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ABSTRACT 


The  recognition  of  a  need  for  a  detailed  assessment  of  the  role  and  timing  of  Human 
Factors  Engineering  (HFE),  in  the  process  of  acquiring  major  naval  weapon  systems,  was 
precipitated  by  the  publication  of  the  report^Human  Factors  Engineering  for  Navy  Ship 
Systems  Acquisitions,  ESSEX  Corporation,  August  1976.  >Three  tasks  were  performed  in 
the  preparation  of  the  present  report:  (1)  The  Navy  Weapon  System  Acquisition  process 
was  defined,  with  supporting  documentation.  Major  acquisition  phases,  milestones,  events 
and  activities  were  identified  and  formatted  into  a  timeline.  (2)  A  comprehensive  review 
of  the  scientific  literature  was  conducted  in  order  to  identify  viable  HFE  methods, 
techniques,  principles  and  data.  These  technologies  were  then  described,  along  with 
methods  of  application  for  each.  (3)  An  extensive  assessment  was  made  of  each 
technology,  in  terms  of  meeting  HFE  requirements,  as  well  as  applicability  and  appropri¬ 
ateness  within  the  acquisition  cycle. 

The  report  is  presented  in  four  sections:  Section  1,  the  Introduction,  provides 
general  background  information  and  defines  the  approach  taken;  Section  2  defines  the 
Navy  Major  Weapon  System  Acquisition  process  and  identifies  HFE  requirements  within 
that  process.  Forty-seven  major  acquisition  events,  activities  and  milestones  and  45 
general  HFE  requirements  are  discussed;  Section  3  provides  descriptions  of  over  70  HFE 
methods  and  techniques,  as  well  as  HFE  principles  and  data  sources.  In  addition,  each 
method  and/or  technique  is  assessed  according  to  its  applicability  to  HFE  requirements 
within  the  acquisition  cycle;  (4)  the  final  section  identifies  HFE  technology  shortfalls  in 
terms  of  addressing  the  HFE  requirements.  It  also  identifies  several  emerging  technolo¬ 
gies  that  are  suitable  to  fill  the  identified  technology  gaps,  v" 

The  effort  was  conducted  under  contract  number  N00024-76-C-6129,  "Human 
Factors  Engineering  Technology  for  Ships,"  for  the  Naval  Air  Development  Center  and  the 
Naval  Sea  Systems  Command  (Code  03416). 
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1.0  INTRODUCTION 


1.1  Background 

In  October  1976,  the  ESSEX  Corporation  published  a  report  for  the  Naval  Sea 
Systems  Command  (Sea  034)  entitled  "HFE  Technology  for  Ship  Acquisition".  The  purpose 
of  the  report  was  to  integrate  Human  Factors  Engineering  (HFE)  Technology  with  the 
Naval  ship  acquisition  process.  This  entailed  addressing  six  separate  considerations: 

1.  The  specific  design  activities,  directives  and  milestones  of  the  acquisi¬ 
tion  process. 

2.  HFE  requirements  as  they  relate  to  each  activity  and  event  of  the 
process. 

3.  The  interrelationships  among  different  HFE  requirements. 

4.  The  available  HFE  technologies  which  are  applicable  to  HFE  require¬ 
ments. 

5.  The  interrelationships  among  HFE  technologies  and  the  activities  in  the 
Naval  ship  acquisition  process. 

6.  The  technology  shorfalls  or  gaps  in  the  technology  base. 

The  report  itseif  was  published  as  two  volumes.  Volume  1  consists  of:  (1)  an 
overview  of  the  phase  activities  in  the  ship  acquisixion  process;  (2)  identification  of  HFE 
requirements  in  each  phase;  (3)  methods  of  satisfying  HFE  requirements  (at  the  time  of 
the  report);  (4)  identification  of  HFE  problem  areas  in  the  ship  acquisition  process;  and 
(5)  for  the  areas  of  manning  and  training,  design  for  operability,  design  for  habitability, 
design  for  maintainability  and  test  and  evaluation.  The  report  describes: 

•  Requirements  and  issues 

•  The  assessment  of  applicable  and  available  HFE  technology 

•  The  identification  of  technology  gaps  and  trends 

•  Recommendations. 

Volume  II  of  the  report  contains  detailed  information  relevant  to  item  5,  above. 

1.2  HFE  for  Major  Naval  Weapon  Systems  Acquisition 

An  outgrowth  of  the  1976  HFE  integration  report  was  the  recognition  that  a 
requirement  existed  for  a  similar  effort,  to  be  directed  towards  all  naval  major  weapon 
system  acquisitions.  In  response  to  this  requirement  a  project  was  initiated  that  has  as  its 
objective  to  survey  and  assess:  (1)  major  milestones  and  events  in  the  Navy  major  weapon 
system  acquisition  process;  (2)  human  factors  engineering  requirements  and  technologies 
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as  they  apply  to  the  acquisition  process;  and  (3)  HFE  technology  shortfalls  related  to  the 
acquisition  process. 


The  approach  taken  to  meet  the  study  objectives  are  described  below  in  terms  of 
four  tasks. 


Task  l  -  Define  the  Navy  Major  Weapon  System  Acquisition  Process. 


•  Identify  the  phases  of  the  major  weapon  system  acquisition  process 

•  Identify  major  milestones,  events  and  activities  in  each  major  phase 

•  Identify  input,  output  and  decision  requirements  for  each  major 
milestone,  event  and  activity 

•  Identify  products  and  information  outputs  for  each  major  milestone, 
event  and  activity 

•  Format  acquisition  cycle  into  a  timeline  with  accompanying  text. 


Task  2  -  Survey  Human  Factors  Engineering  Technoloe 


•  Survey  available  and  emerging  HFE  methods,  techniques,  principles 
and  data 

•  Classify  technologies 

-  descriptive 

-  analytic 

-  design-oriented 

-  evaiuation/assessment-oriented 

-  integrative 

•  Describe  technology  in  terms  of: 

-  objective 

-  source 

-  application 

-  state  of  development 

-  problems  identified 

•  Describe  each  technology  method  of  application 


Task  3  -  Assess  and  Integrate  HFE  Technology  With  the  Acquisition  Process. 


•  Identify  HFE  requirements  at  each  step  of  the  acquisition  cycle 

•  Develop  and  apply  criteria  for  technology  assessment  according  to: 

-  usability 

-  impact  on  system  design 

-  cost 

-  alternative  technologies 

-  potential  for  computerization 

-  standardization 

•  Identify  HFE  inputs  to  products  of  the  acquisition  cycie 

•  Identify  acquisition  cycle  information  inputs  to  HFE  activities 
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•  Identify  HFE  windows  (time  periods)  wherein  required  events  must  be 
completed  with  indications  of  consequences  of  failure 

•  Format  acquisition  cycle  and  HFE  design  process  into  a  timeline 

•  Identify  HFE  technology  shortfalls 

Task  4  -  Prepare  Report.  This  task  required  the  consolidation  and  codification  of 
information  gathered  in  the  previous  tasks.  Data  in  the  report  is  presented  in  four 
sections,  as  follows: 

1.  Introduction 

2.  Navy  major  weapon  system  acquisition  process  integration  with  Human 
Factors  Requirements 

•  Acquisition  process  (with  major  phases,  milestones,  activities) 

•  HFE  process  and  requirements 

•  HFE  inputs  to  the  acquisition  cycle 

•  Acquisition  cycle  major  event  and  activities  inputs  to  H' 
process 

3.  Survey  of  the  applicability  of  HFE  methods,  techniques,  principles  a, 
data  to  specific  HFE  requirements  within  the  acquisition  process. 

4.  Statements  of  identified  technology  shortfalls,  identification  of  emerg¬ 
ing  HFE  techniques  and  methods  suitable  to  fill  technology  shortfalls. 
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2.0  NAVY  WEAPON  SYSTEMS  ACQUISITION  PROCESS 


2.1  Formal  Acquisition  Policy 

In  April,  1976,  the  Office  of  Management  and  Budget  (OMB)  released  Circular 
Number  A- 109  which  establishes  policies  for  acquisition  of  major  systems.  This  document 
details  the  responsibilities  and  issues  to  be  addressed  in  acquiring  systems.  OMB  Circular 
Number  A- 109  is  provided  in  Appendix  B. 

Three  basic  documents  direct  the  Navy  (and  all  other  services)  in  implementing  the 
requirements  of  A- 109.  These  are  Department  of  Defense  (DoD)  Directives  5000.1, 
5000.2  and  5000.3. 


DoD  Directive  5000.1,  "Major  Systems  Acquisitions"  (January  1977),  provides  basic 
system  acquisition  policy  for  systems  costing  over  $75  million  research,  development,  test 
and  evaluation  (RDT&E)  or  over  300  million  procurement  dollars.  Acquisition  policy  as 
set  forth  in  the  Directive  is  summarized  briefly  as  follows: 


•  Acquisition  is  a  sequence  of  phases  initiated  by  approval  of  a  mission  need. 


DoD  components  (Army,  Navy,  Air  Force)  are  to  analyze  and  identify 
mission  needs,  and  to  develop  systems  which  fulfill  those  needs. 


The  Secretary  of  Defense  (SECDEF)  renders  decisions  regarding  program 
commitments  (to  initiate  programs,  direct  program  funding).  Four 
SECDEF  decision  points  are  identified: 

-  Milestone  0  -  Program  initiation 

-  Milestone  I  -  Demonstration  and  validation 

Milestone  n  -  Full-scale  engineering  development 
Milestone  III  -  Production  and  deployment 


The  Milestone  0  decision  requires  that  a  mission  need  is  demonstrated  in  a 
document  called  the  Mission  Element  Need  Statement  (MENS).  The 
Milestone  I  decision  (to  procede  to  the  next  phase)  is  based  on  recommen¬ 
dations  documented  in  the  Decision  Coordinating  Paper  (DCP).  The 
Milestone  II  and  III  revisions  are  based  on  updated  revisions  of  the  DCP. 


•  Mission  needs  are  to  be  satisfied,  where  feasible,  with  existing  hardware 
and  software. 


•  Test  and  evaluation  is  to  be  commenced  as  early  as  possible. 

•  Alternate  maintenance  concepts  are  part  of  logistic  support  planning. 

•  Human  engineer  factors  are  to  be  included  as  constraints  in  system  design. 
"The  integration  of  the  human  element  and  system  shall  start  with  the 
initial  concept  studies  and  refined  as  the  system  program  progresses  to 
form  the  basis  for  personnel  selection  and  training,  training  devices, 
simulators  and  planning  related  to  human  factors." 


DoD  Directive  5QQ0.2,  "Major  System  Acquisition  Process"  (January  1977),  estab¬ 
lishes  the  process  by  which  major  systems  are  acquired.  It  establishes  that  the  SECDEF 
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will  exercise  direction  and  control  of  acquisition  programs  through  four  milestone 
decisions  concerning  further  program  conduct.  It  further  establishes  advisory  councils  to 
review  DCPs  and  make  recommendations  concerning  program  direction  and  continuation. 
The  Defense  System  Acquisition  Review  Council  (DSARC)  (Tri  Service)  and  Department 
of  the  Navy  System  Acquisition  Review  Council  (DNSARC)  are  so  chartered  as  the 
organizations  for  Navy  acquisitions. 

DoD  Directive  5000.2  also  describes  required  documentation  to  support  DSARC  and 
SECDEF  acquisition  program  reviews,  recommendations  and  decisions;  these  include  the 
Mission  Element  Need  Statement  (MENS)  and  the  DCP.  The  MENS  is  used  by  the  SECDEF 
at  the  initial  decision  point.  Milestone  0  (program  initiation),  and  ultimately  becomes  part 
of  the  DCP  and  DSARC  process. 

The  directive  also  schedules  program  reviews  and  SECDEF  decision  making.  Four 
program  reviews  (milestones)  are  called  for  by  DoD  Directive  5000.2. 

•  Milestone  0  -  Program  initiation 

•  Milestone  I  -  Demonstration  and  validation 

•  Milestone  II  -  Full-scale  engineering  development 

•  Milestone  HI  -  Production  and  deployment 

At  Milestone  0,  the  SECDEF  makes  the  decision  concerning  program  initiation  by 
reviewing  the  MENS;  at  Milestones  I,  II  and  III,  the  SECDEF  makes  program  decisions 
utilizing  the  DCP  and  DSARC  recommendations.  The  activities  conducted  prior  to  each 
of  the  milestones  are  depicted  in  Figure  1. 

DoD  Directive  5000.3,  "Test  and  Evaluation"  (1977),  determines  that  all  systems  will 
be  subject  to  test  and  evaluation  (T&E)  and  will  be  part  of  the  DSARC  and  SECDEF 
Milestone  decisions. 

Four  general  principles  are  set  forth  by  the  directive: 

1.  T<kE  shall  be  commenced  as  early  as  possible  in  the  acquisition  cycle, 
and  shall  be  conducted  throughout. 

2.  Acquisition  schedules  will  be  based  on  accomplishing  T&E  Milestones. 

3.  T<JcE  of  existing  or  modified  equipment  may  be  performed  prior  to  the 
initiation  of  a  new  system  development,  in  order  to  help  define  military 
need  and  estimate  military  utility  of  the  new  system. 

4.  T&E  activities  shall  consider  environmental  issues  and  provide  assess¬ 
ments  for  review  as  early  as  possible  in  the  test  planning  cycle. 

The  directive  also  requires  that  integrated  T<5cE  plans  be  established  and  kept 
current  with  all  system  TdcE  efforts  and  schedules.  This  plan  is  to  be  established  as  early 


2-2 


as  possible  in  the  acquisition  cycle,  and  must  be  completed  prior  to  Milestone  II.  Further 
requirements  for  T<5cE,  as  part  of  the  DSARC  process,  are  stated.  Briefly,  these  are: 


•  The  DCP  at  Milestone  I  will  identify  critical  questions  and  areas  of 
risk  to  be  resolved  by  TicE 

•  The  DCP  at  Milestone  II  will  provide  results  of  T<5cE  efforts  to  that 
date  and  update  critical  questions  and  areas  of  risk 

•  DSARC  will  review  TJcE  results  prior  to  making  recommendations  to 
the  SECDEF  at  Milestone  in. 

The  above  documents  provide  direction  during  the  acquisition  process.  Another 
document,  MIL-H-46855,  "Human  Engineering  Requirements  for  Military  Systems,  Equip¬ 
ment  and  Facilities,  is  directed  specifically  at  the  role  of  HE  in  the  acquisition  process. 

This  specification  states  that  human  factors  program  requirements  are  to  include: 

•  Defining  and  allocating  system  functions.  Human  Factors  Engi¬ 
neering  principles  and  criteria  are  to  be  applied  to  allocate  system 
functions  to 

-  automatic  operations/maintenance 

-  manual  operation/maintenance  or 

-  a  combination  of  manual /automatic  operation/maintenance 

•  Information  flow  and  processing  analysis 

•  Estimates  of  potential  operator /maintainer  processing  capabilities. 

Roles  to  be  identified  for  humans  such  as 

-  operator 

-  maintainer 

-  programmer 

-  decision  maker 

-  communicator 

-  monitor 

are  required.  Estimates  concerning  load,  accuracy,  rate,  etc.,  are 
also  to  be  identified 

•  Equipment  identification.  HFE  principles  and  criteria  are  to  be 
incorporated  into  the  identification  or  selection  of  equipment  which 
are  to  be  operated/controlled/maintained  by  man. 

•  Task  analysis.  To  be  conducted  and  applied  to  design  decisions, 
analysis  of  manning  levels,  equipment  procedures,  etc. 

•  Analysis  of  critical  tasks.  Task  analysis  (above)  extended  to  analysis 
of  critical  tasks  to  identify,  for  example: 

-  information  required  by  man 

-  information  available  to  man 

-  information  evaluation  process 

-  decision  reached 

-  action  taken 

-  body  movements 

-  tool  required 

-  job  performance  aids  (JPA)  required 

•  Loading  analysis.  Crew/individual  workload  analysis  is  to  be  applied 
and  compared  to  performance  criteria. 
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•  Preliminary  system  and  subsystem  design.  HFE  principles  and  data 
are  to  be  applied  to  system/subsystem  design.  MIL-STD-1472  is  to 
be  complied  with. 

•  Detailed  design.  As  above. 

•  Studies,  experiments,  laboratory  tests.  Research  is  to  be  conducted 
to  resolve  man/machine  trade-off  problem  areas  and  other  HFE  and 
life  support  problems. 

•  Mock-ups  and  models.  Mock-ups  (3-D)  to  be  constructed  as  an  HFE 
design  evaluation  tool. 

•  Dynamic  simulation  (as  required  for  HFE  design). 

•  Design  drawings. 

•  Workspace  environment.  This  would  include 

-  atmospheric  conditions 

-  weather  and  climate 

-  bodily  acceleration 

-  noise 

-  safety  (handholds,  etc.) 

•  Test  and  evaluation.  Planning,  implementation  and  failure  analysis. 

Figure  2  shows  functional  relationships  of  MIL-H-46855  Human  Factors  Requirements 
(adapted  from  Geer,  1976). 


2.2  Requirements  Throughout  the  Acquisition  Ode 

HFE  requirements  and  the  Navy  major  weapon  system  acquisition  cycle  are 
presented  in  Figure  1.  Requirements,  inputs,  outputs  and  uses  for  each  HFE  step  are 
shown  in  Table  1.  The  schedule  of  applying  HFE  requirements  is  presented  in  Figure  3. 
The  following  text  describes,  for  each  phase,  HFE  requirements  and  major  acquisition 
steps. 

There  are  five  acquisition  phases,  each  leading  to  a  program  milestone.  These  are: 
(1)  feasibility/analysis;  (2)  program  initiation;  (3)  demonstration  and  validation;  (4)  full- 
scale  engineering  development;  and  (5)  production  and  deployment. 
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•  Input  to: 

-  task  analysis 

-  operational  sequence  analysis 
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2.2.1  Requirements  Uo  to  Milestone  I 

2.2.1. 1  Feasibility/ Analysis  Phase  -  Summary.  In  this  phase,  the  major  objective 
and  activity  is  the  response  to  the  identification  and  definition  of  a  mission  need.  Once  a 
need  (mission  requirement)  has  been  identified,  some  exploratory  research  may  be 
performed  in  order  to  develop  alternative  methods  of  satisfying  the  mission  requirement. 
Phase  activities  and  the  MENS  are  evaluated  by  the  SECDEF  (Milestone  0)  and  a  decision 
is  made  to  halt  further  effort,  to  require  additional  mission  area  analysis,  alternate 
concept  development,  etc.,  or  to  proceed  to  the  next  acquisition  phase. 

2.2. 1.2  Feasibility /Analysis  Phase  -  Detailed  Discussion.  Either  a  technological 
development  which  counters  a  known  threat  or  the  recognition  of  a  tactical  threat  may 
initiate  the  development  of  a  new  weapon  system.  In  the  first  case,  the  uncovering  of 
improved  propulsion  systems,  sensors,  weapons,  etc.,  by  industry  or  government  agencies, 
may  initiate  the  development  of  a  new  weapon  system  which  counters  a  known  threat.  In 
the  second  case,  the  discovery  of  combat/weapon  systems  possessed  by  potentially  hostile 
forces  may  be  evaluated  as  a  military  threat,  such  that  the  development  of  new 
combat/defensive  systems  may  eventually  be  called  for  in  order  to  counter  that  threat. 
Tactical  threats  may  be  identified  by  analysis  of  relative  force  levels,  intelligence 
information,  system/mission  effectiveness  models,  etc. 

Mission  Element  Need  Statement 

With  the  identification  of  a  mission  need,  the  MENS  is  prepared.  As  called  out  in 
DoD  Directive  5000.2,  the  MENS  is  a  required  document  which  is  to  state: 

•  Mission  area  and  need  in  terms  of  mission  tasks  to  be  performed 

•  Projected  threat  assessment  through  the  time  frame  in  which  a 
capability  is  required 

•  Existing  capabilities  to  accomplish  the  mission 

•  Need  in  terms  of  existing  capability  deficiency 

•  Known  constraints  to  solutions  (cost,  standardization  with  NATO, 
time  frames,  etc.) 

•  Impact  of  lack  of  capability 

•  A  plan  for  the  identification  and  exploration  of  alternative  systems 

The  MENS  is  essentially  a  short  statement  of  a  present  or  projected  threat  and  proposed 
solutions  to  counter  that  threat. 

The  MENS  is  forwarded  to  both  the  Office  of  the  Secretary  of  Defense  (OSD)  and 
the  Office  of  the  Joint  Chiefs  of  Staff  (OJCS)  for  review  and  comments.  When  the  review 
has  been  completed,  the  MENS  and  comments  are  presented  to  the  Secretary  of  Defense. 
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The  SECDEF  decision  for  program  initiation  is  based  upon  the  MENS  and  attached 
comments  and  position  papers.  The  SECDEF’s  signature  (at  Milestone  0)  initiates  the 
conceptual  phase  of  system  acquisition  which  consists  of  identifying  and  exploring 
alternate  solutions  to  the  stated  threat. 

2.2. 1.3  Program  Initiation  Phase-Summary.  Major  activities  of  this  phase  are: 
reestablish  mission  need,  survey  available  technology  in  order  to  identify  areas  of 
technology  inadequacy  for  proposed  systems,  begin  the  definition  of  an  acquisition 
strategy,  and  prepare  and  issue  documentation  required  for  the  Milestone  I  decision.  The 
SECDEF  decision  at  Milestone  I  marks  the  initial  involvement  of  DSARC.  The  SECDEF 
can,  at  this  time,  cancel  the  program,  request  continued  phase  activities  or  begin  the 
Demonstration  and  Validation  Phase. 

2.2.1. 4  Program  Initiation  Phase  -  Detailed  Discussion.  SECNAV  Instruction  5000.1 
(which  implements  DoD  5000.1  for  the  Navy)  states  that  the  conceptual  design  phase  shall 
be  directed  towards  specifying  a  broad  range  of  performance  and  operating  characteris¬ 
tics  of  the  system.  In  beginning  the  conceptual  (program  initiation)  phase,  alternate 
solutions  are  developed  by  the  identification  of  whatever  required  technology  advances 
are  necessary  to  complete  a  weapon  systems  suit.  Where  technology  is  insufficient,  it  is 
termed  a  shortfall.  A  variety  of  technologies  may  have  to  be  assessed,  including  guidance 
technology,  propulsion  technology,  navigation,  etc. 

Science  and  Technology  Objectives 

Once  these  technology  shortfalls  are  identified,  they  are  formulated  into  Science 
and  Technology  Objectives  (STOs).  STOs  are  statements  of  capabilities  required,  but  not 
yet  existent,  for  the  proposed  combat  system.  STOs  are  formulated  from  previously 
identified  required  technological  advances  and  the  tactical  requirements  of  the  weapon 
system.  These  objectives  may  lead  to  the  upgrading  of  existing  systems  or  subsystems 
(radar  sensitivity,  range,  as  an  example),  or  complete  redesign  or  development  of  a  system 
to  effect  compatibility  with  the  point  system. 

Preliminary  Human  Engineering  Analysis 

Sufficient  progress  at  this  point  will  have  been  made  to  initiate  a  formal  Human 
Factors  Engineering  (HFE)  effort  in  the  system  acquisition.  This  first  step,  the 
Identification  of  Operational  Conditions,  will  serve  to  familiarize  the  HFE  analyst  with 
the  proposed  system  and  will  lay  a  foundation  for  future  HFE  analysis.  The  data  collected 
consists  of  use  and  tactical  conditions  which  are  expected  to  be  experienced  by  the 
weapon  system.  Use  conditions  such  as: 
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•  Operational  modes 

•  States  of  readiness 

•  Modes  of  communication 

•  Emergency/contingency  condition 

may  be  collected  by  the  HF  engineer.  Tactical  conditions  to  be  identified  are: 

•  Enemy  characteristics/capabilities 

•  Enemy  behavior  responses 

•  Own  forces  characteristics/capabilities 

•  Own  forces  behavior /responses 

•  Own  system  characteristics/capabilities 

This  step  may  serve  as  the  initial  foundation  upon  which  subsequent  HFE  steps  may 
be  structured  and  may  suggest  the  level  of  effort  that  will  be  required  to  integrate  HF  in 
the  system  design,  constraints  on  human  performance,  HFE  test  and  evaluation  and 
general  planning  direction. 

Following  the  identification  of  operational  and  tactical  conditions,  the  HF  engineer 
can  initiate  an  analysis  of  similar  systems  to  be  applied  at  the  total  system  or  subsystem 
level.  Of  great  utility  for  the  HF  engineer,  this  analyses  will  afford  an  operational  and 
design  baseline  from  which  improvements  in  the  developing  system  can  be  made  and 
measured;  further,  it  will  point  out  design  problem  areas  that  can  be  avoided  in  the 
developing  system. 

Systems  similarities  can  extend  to: 

•  Missions 

•  Operations 

•  Mission  major  events 

•  System  functions,  etc. 

From  these  similarities,  operator  and  operational  data  can  be  gathered,  i.e., 

•  Functional  allocations 

•  Operational  performance  histories 

•  Operational  timelines 

•  Operator  workloads 

•  Human  factors  design  problem  identification 

The  requirement  for  this  step  lies  in  the  fact  that  a  great  deal  of  data  may  (and 
probably  does)  exist.  These  data  may  be  used  in  nearly  all  subsequent  HF  design  activities 
such  as  functional  allocations,  requirements  analysis,  workspace  design,  etc. 


With  the  review  of  the  MENS  by  the  Secretary  of  Defense  and  the  establishment  of 
STOs,  advanced  systems  concept  development  is  commenced.  Some  systems  and 
subsystems  will  be  identified  for  the  combat  system  and  explorations  of  alternate 
concepts  that  will  satisfy  the  STOs  continue.  Total  systems  concepts  may  be  available 
which  will  show,  to  some  extent,  factors  defined  to  greater  detail,  equipment  constraints 
such  as  manning  levels,  and  indications  of  human  operator  requirements. 

Concurrent  with  advanced  system  concept  development  is  the  HFE  analysis  of 
system  functions  which  is  a  fundamental  step  in  the  design  process.  As  the  system  takes 
form,  an  increasingly  more  detailed  description  of  the  system  can  be  established  and 
analyzed.  Through  this  step,  functional  analysis  will: 

•  Identify  missions/mission  problems  and  operations 

•  Establish  mission/operation  priorities 

•  Identify /establish  mission/operations  major  events 

•  Identify  mission  functions,  subfunctions,  etc. 

•  Analyze  system  functions  and  subfunctions, 

This  material  is  required  for  the  very  basic  HFE  design  steps  of  functional  allocations  and 
procedures  generation. 

An  Environmental  Analysis  is  then  performed  to  identify  operational  conditions 
affecting: 

•  Visibility 

•  Communications 

•  Operations 

•  Safety 

•  Work  performance 

The  HF  engineers  must  identify  these  conditions  and  their  potential  constraints  upon 
human  performance.  The  HF  analyst  will  design  to  constraints  imposed  by  the  degrada¬ 
tion  of  equipment  operation  due  to  environmental  factors. 

A  Requirements  Analysis  is  now  performed  in  which  requirements  for  each  function 
and  subfunction  by  mission  and  operational  conditions  are  identified.  Information 
requirements  for  each  function  are  identified  which  include: 

•  Information  required 

•  Source  of  information 

•  Accuracy  requirements 
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•  Currency  requirements 

Examples  of  performance  requirements  identified  are: 

•  Accuracy  limits 

•  Response/performance/compietion  time  limits 

•  Energy  expenditure 

•  Limits  on  error  rates 

•  Frequency  of  occurance 

Decision  requirements  are  also  identified,  e.g., 

•  Decisions  to  be  made 

•  Options 

•  Decision  rules/risks 

With  the  availability  of  such  data  as  system  functions  and  requirements,  environ¬ 
ment  and  operational  conditions,  an  allocation  of  functions  can  be  performed  in  which 
performance  of  functions  is  allocated  to  men  or  machines,  or  among  different  men. 
Several  allocation  schemes  are  usually  selected  according  to  such  constraints  and  criteria 
as: 

•  Costs 

•  Convention 

•  Command  decisions 

•  Relative  man/machine  capabilities 

•  Relative  man/machine  reliabilities 

•  Operationai/engineering  complexity 

•  Level  of  system  automation 

•  Manpower  availability 

Operator  performed  functions  are  then  further  allocated  to  sets  of  related  func¬ 
tions,  thereby  establishing  the  rudiments  of  an  individual  operator's  job.  This  step  is 
critical  to  an  HFE  design  process  if  viable  allocation  schemes  are  to  be  derived. 
Otherwise,  inadequate  man/machine  and  man/man  function  allocations  will  degrade 
system  performance. 

DSARC  Process 

Within  the  time  frame  of  the  development  of  the  STOs,  the  Operational  Require¬ 
ment  (OR)  is  prepared.  The  OR  is  a  statement  of  the  operational  need  of  the  new  weapon 
system  and  initiates  the  conceptual  effort  to  meet  the  stated  operational  need. 
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The  initial  OR  may  have  undefined  areas,  but  is  updated  prior  to  the  next  program 
decision.  Contained  in  the  OR  is  a  cost  constraint,  a  cost  target  that  is  estimated  to  be 
near  that  of  the  actual  acquisition.  An  important  aspect  of  the  OR  is  the  establishment 
of  the  Developmental  Proposal  (DP),  describing  (1)  the  technical  approach  which  will 
satisfy  the  operational  requirement.  The  DP  provides  alternate  approaches  and  develop¬ 
ments  that  are  applicable  to  fulfilling  the  OR,  (2)  an  economic  analysis  and  relative 
benefits  of  alternate  technical  approaches;  and  (3)  a  recommendation  for  the  technical 
approach. 

In  the  time  frame  of  the  development  of  the  DP,  the  Decision  Coordinating  Paper 
(DC?)  is  generated.  The  DCPs  principal  purpose  is  to  support  the  SECDEF  and  Defense 
System  Acquisition  Review  Council  (DSARC)  in  determining  program  continuation.  The 
DCP  is  to  contain  (as  per  DoDINST  5000.2): 

•  An  approved  MENS 

•  Information  updating  MENS 

•  Alternate  program  descriptions 

•  Summary  of  acquisition  strategy 

•  Short/long  term  business  planning  strategy 

•  Management  plan 

•  Technical  risk  estimates 

•  Test  and  evaluation  planning  status 

The  DCP  and  DP  are  used  by  SECDEF  and  DSARC  in  rendering  ail  subsequent  major 
milestone  decisions.  DoDINST  5000.2  details  the  issues  to  be  addressed  at  each  DSARC 
decision  point;  generally  these  include: 

•  Reaffirmation  of  mission  need 

•  Upuated  threat  assessments 

•  Alternate  strategies  to  be  considered 

•  Operational  and  logistics  considerations 

•  Acquisition  strategies 

•  Risk  estimates 

•  Test  and  evaluation  mast/r  plans  (TEMPs) 

The  DCP  is  the  principle  source  of  these  data  and,  therefore,  is  updated  throughout 
the  weapon  system  acquisition  cycle  and  reviewed  at  each  DSARC  decision  point. 

At  the  DSARC  I  (or  Milestone  I),  the  DCP  additionally  states: 

•  whether  the  validation  phase  is  to  be  entered  with 
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-  several  system  concepts 

-  a  single  system  concept,  or 

-  involvement  of  alternate  subsystems  only,  design  not  to  be 
conducted  at  the  total  svstem  level,  or 

•  whether  to  proceed  directly  to  full-scale  engineering  development. 

At  Milestone  I  the  DCP  will  contain  a  Technology  Assessment  Annex  (TAA)  which 
identifies  areas  of  technological  risks  and  defines  plans  for  addressing  these  risks. 

At  Milestone  I,  the  DCP  is  forwarded  to  DSARC  for  review  and  action.  DSARC 
recommendations  are  then  forwarded  to  the  Secretary  of  Defense  for  final  decision.  At 
this  point  in  the  HFE  design  process,  system  functions  have  been  (and  are  being)  identified 
and  analyzed,  environmental  conditions  identified,  similar  systems  analyzed,  preliminary 
manning  estimates  made  and  functions  allocated  (initially)  between  men  and  machAes  and 
among  operator  stations.  The  operational  system  is  beginning  to  take  form.  In  order  to 
document,  solidify  and  develop  crew  requirements,  position  descriptions  are  formulated. 
This  generally  entails  a  narrative  description  of  the  general  duties  of  a  station,  i.e., 
operator  roles  and  responsibilities  and  operational  constraints.  Identification  of  skills  and 
knowledge  for  each  position  can  be  initiated. 

HFE  Job/Task  Analysis 

Position  descriptions  will  be  used  in  subsequent  analysis  to  provide  a  framework 
from  which  to  develop  stations  and  to  modify /document  prior  efforts. 

The  individual  tasks  for  each  operator  are  identified  (through  analysis  of  tasks  and 
task  requirements);  in  addition,  requirements  for  performing  each  task  are  determined.  In 
conducting  the  analysis,  functions  are  further  allocated  (or  reallocated)  to  individual 
operators.  Functions  become  tasks,  or  are  broken  down  to  individual  tasks  comprising  a 
function,  and  a  sequential  task  index  for  each  position  is  developed. 

For  each  task,  requirements  of  the  following  sort  are  identified: 

•  Activity 

•  Performance  time 

•  Information/'communication 

•  Controls/displays 

•  Constraints  on  task  performance 

Task  criticalities  and  priorities  may  also  be  determined  in  the  course  of  the  analysis. 

With  the  availability  of  functional  analysis  and  allocations,  task  sequences,  and  task 
requirements  data,  operational  sequences  between  and  among  operating  stations  can  be 
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analyzed.  In  so  doing,  links  between  operators  and  types  of  links  (electronic,  verbal, 
visual,  etc.)  are  identified  and  the  syncronization  and  phasing  of  events  are  examined. 
These  analyses  are  required  in  order  to  examine  and  evaluate  function  and  task 
allocations,  workloading,  operational  and  communications  links,  etc. 

2.2.2  Requirements  to  Milestone  II 

2.2.2.1  Demonstration  and  Validation  Phase  -  Summary.  With  the  SECDEF's 
approval  to  enter  this  phase,  the  following  has  been  accomplished: 

•  Formation  of  alternate  weapon  system  concepts 

•  Identification  of  subsystems  targeted  for  advanced  development 

•  Mission  need  has  been  defined 

•  Acquisition  strategies  and  plans  have  been  developed  and  approved 
Tne  activities  and  objectives  of  the  demonstration  and  validation  phase  are  to: 

•  Conduct  preliminary  design 

•  Establish  a  formal,  detailed  management  plan 

•  Establish  a  test  and  evaluation  management  plan 

•  Establish  an  integrated  logistics  support  plan 

•  Prepare  Requests  for  Proposals  for  system/subsystem  development 

•  Construct  prototypes  of  systems  and/or  subsystems  for  technical 
evaluations,  and 

•  Prepare  for  the  Milestone  II  decision. 

2.2.2.2  Demonstration  and  Validation  Phase  -  Detailed  Discussion 

Acquisition  Management  and  Planning  Policy 

In  beginning  the  validation  phase,  project  teams  are  designated  and  the  Program 
Master  Plan  (PMP)  is  established.  This  is  a  basic  planning  document  prepared  by  the 
Program  Manager  (PM)  which  itemizes  the  responsibilities  of  participating  organizations 
(contractors  and  government  organizations).  It  sets  forth  plans,  schedules,  costs  and 
scope  of  work  for  each  participating  organization.  Two  important  considerations  of  the 
PMP  are  test  and  evaluation  plans  and  integrated  logistics  support  plans. 

SECNAV  Instruction  5000.1  states:  "Integrated  logistics  support  effort  shall  be 
conducted  as  an  integral  part  of  the  acquisition  process  and  pursued  to  ensure  realistic 
application  of  ILS  considerations."  It  further  states:  "The  purpose  of  ILS  is  to  promote 
development  of  hardware  which  is  not  only  technically  excellent,  but  cost  effective, 
reliable,  easily  maintained  and  operated,  and  able  to  be  realistically  supported  when 


•••■  ••  t 


2-27 


delivered  for  operational  use."  As  part  of  the  PMP,  the  Integrated  Logistics  Support  (ILS) 
Master  Plan  is  established.  ILS,  as  defined  by  NAVMATINST  4000. 20B,  is  "a  composite  of 
ail  the  support  considerations  necessary  to  assure  the  effective  and  economical  support  of 
systems/equipments  for  their  life  cycle.  It  is  a  integral  part  of  system /equipment 
acquisition  and  operation  and  is  characterized  by  harmony  and  coherence  among  all  the 
logistics  elements."  The  principal  elements  related  to  the  overall  system/equipment  life 
cycle,  includes: 

•  Maintenance  planning 

•  Support  and  test  equipment 

•  Supply  support 

•  Transportation  and  handling 

•  Technical  data 

•  Facilities 

•  Personnel  and  training 

•  Logistic  support  resource  funds 

•  Logistic  support  management  information 

The  ILS  Plan,  then,  is  one  in  which  logistics  concepts,  techniques  and  policies  are 
implemented  to  assure  "the  effective  economical  support  of  a  system /equipment  during 
its  life  cycle  and  details  what  ILS  tasks  are  to  be  accomplished,  who  is  responsible,  how 
they  are  to  be  accomplished  and  when." 

Typical  elements  in  the  ILS  plan  are  as  follows: 

•  System  /equipment  description 

•  The  assigned  ILS  manager 

•  Management  plans 

•  Personnel  and  training  requirements 

•  Supply  support  plans 

•  Test  equipment 

About  the  time  of  DSARC  I,  the  Test  and  Evaluation  Master  Plan  (TEMP)  has  been 
established.  The  TEMP  is  a  document  prepared  by  the  program  manager  and  is  used  both 
as  part  of  the  DSARC  decision  process  and  as  a  management  plan  for  Test  and  Evaluation 
(T5cE).  The  TEMP  identifies  the  testing  to  be  performed  before  the  DSARC  II  5c  III 
reviews. 

Testing  is  performed  on  both  a  component  level  and  a  systems  level.  Component 
testing  consists  of  demonstration  and  validation  testing  of  components  intraoperability, 
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maintenance  requirements,  etc.  Systems  level  TJcE  consists  of  Development  Test  and 
Evaluation  (DT3cE)  and  Operational  Test  and  Evaluation  (OT<5cE).  DT3cE  has  as  its  purpose 
to: 

•  Demonstrate  that  the  engineering  design  is  complete 

•  Demonstrate  that  design  risks  are  minimized 

•  Demonstrate  that  the  system  meets  operational  requirements,  and 

•  Estimate  military  utility  of  the  system  when  introduced 

The  level  of  developmental  testing  "shall  be  adequate  to  ensure:  that  engineering  is 
reasonably  complete;  that  all  significant  design  problems  (including  comparability,  intra¬ 
operability,  reliability,  maintainability  and  logistical  considerations)  have  been  identi¬ 
fied".  (DoD  INST  5000.3.) 

The  OT&E  serves  the  purposes  of  estimating: 

•  Military  utility  of  the  system 

•  Operational  effectiveness,  and 

•  Operational  suitability  (as  in  DT&E  with  the  added  consideration  of 
training  requirements) 

and  in  providing  information  on  organizations,  personnel  requirements,  doctrine  and 
tactics. 

HFE  Involvement  in  Planning  Policy 

With  the  development  of  the  TEMP,  the  HFE  step  to  identify  HFE  T&E  problem 
areas  is  initiated.  This  step  calls  for  the  identification  of  areas  in  which  HFE  design 
problems  may  become  evident  in  the  weapon  system.  These  areas  are  related  to 
performance/system  operability,  environment,  information,  communications  and  manning 
and  training. 

The  requirements  of  the  step  are  to:  identify  potential  problem  areas  in  order  that 
the  human  engineer  may  specifically  address  them  during  equipment  design  and  also  to 
plan  formal  evaluation  of  these  areas  as  part  of  the  test  and  evaluation  process.  The 
results  of  this  step  can  be  formulated  into  several  test  requirements  and  input  to  the  Test 
and  Evaluation  Master  Plan  (TEMP),  thereby  incorporating  HFE  .  o  the  overall  test  plans. 
A  further  result  of  this  step  is  the  development  of  an  HFE  Test  and  Evaluation  Plan  which 
will: 

•  Itemize  HFE  issues  to  be  tested 

•  Identify  areas  where  specific  issues  will  be  emphasized  in  testing 

•  Provide  test  schedules 
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As  ILS  plans  and  requirements  are  developed  and  defined,  the  maintenance  philos¬ 
ophy  of  the  weapon  system  will  evolve.  The  maintenance  philosophy  should  cover  such 
areas  as  overhaul  cycles,  levels  of  organizational  or  depot  repair,  system  performance 
monitoring  schemes,  training  and  test  equipment,  standardization  of  components,  planned 
and  corrective  maintenance  schemes  (remove  and  replace,  remove,  repair  and  replace). 

NAVMAT  Instruction  4000. 20B  calls  for  a  level  of  repair  (LOR)  analysis  to  be  made 
for  all  Naval  material  being  acquired  for  the  operational  inventory.  LOR  analysis  is 
defined  as  an  economic  and  non-economic  evaluation  used  to  establish  the  maintenance 
level  at  which  an  item  will  be  replaced,  repaired  or  discarded.  Non-economic  LOR 
criteria  are  cited  as  being: 

•  Safety 

•  Vulnerability 

•  Survivability 

•  Mission  success 

-  criticality 

-  effectiveness 

•  Manning 

•  Human  Factors 

-  special  skills 

•  Deployment  mobility 

•  Policy  (specifications) 

•  Technical  feasibility  of  repair 

•  Special  transportation  factors 

These  data  can  be  used  to  identify  the  roies  of  maintenance  technicians  for  each 
maintenance  function,  therein  providing  necessary  information  for  developing  mainte¬ 
nance  3PA  concepts,  performing  requirements  analysis,  task  analysis  and  for  training 
system  development. 

Initial  Training  System  HFE  Requirements 

Data  from  the  Task  Requirements  analysis,  operational  sequence  analysis,  mainte¬ 
nance  philosophy  are  used  to  identify  job  performance  aid  (3PA)  requirements.  This  step 
represents  the  first  eifort  towards  the  development  of  the  training  program  for  the 
weapon  system. 

Several  steps  are  involved  in  determining  3PA  requirements: 

•  Identify  the  information  to  be  conveyed 

•  Identify  information  type 
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-  procedural  (operational  or  maintenance) 

-  instructional 

-  computational 

-  decision  making 

•  Identify  characteristics  that  affect  3PA  requirements 

•  Identify  those  units  or  sequences  of  information  that  are  best  learned 
or  presented  by  JPAs 

The  JPA/training  decision  is  made  (in  part)  by  the  identified  characteristics  of  the  tasks 
or  information  to  be  learned,  i.e.,  ease  of  learning,  requirements  for  branching  steps, 
number  of  similar  tasks,  frequency,  etc.  For  example,  for  infrequently  required  tasks  that 
require  a  great  many  branching/decision  steps,  JPAs  are  called  for,  but  for  frequently 
required  and  easily  learned  tasks,  training  is  a  more  appropriate  tool. 

This  JPA/training  decision  leads  to  the  development  of  3PA  concepts,  the  identifi¬ 
cation  of  skill/knowledge  requirements,  training  objectives,  etc.,  and  greatly  facilitates 
training  program  definition  and  development.  JPAs  can  then  be  developed  by  first 
identifying  constraints  and  developing  concepts  involving: 

•  Computer-generated  displays 

•  Manuals 

•  Special  guides 

Feasible  concepts  can  then  be  selected  for  trade-offs. 

Performed  concurrently  with  the  identification  of  JPA  requirements  is  an  analysis 
of  required  skills  and  knowledges  which  will  provide  the  Human  Factors  Engineer  with 
data  concerning  the  number  and  complexity  of  skills  required  and  the  magnitudes  of 
knowledge  requirements.  Skill  requirements  for  each  position  are  assembled  according  to 
skill  levels  required,  performance  standards,  criticality  and  similarity  to  other  skills. 
Knowledge  requirements  for  each  position  are  assembled  by  type  (diagnostic,  procedural, 
etc.),  learning  difficulty  and  criticality. 

Initial  HFE  Maintainability  Design  Requirements 

Upon  the  identification  of  a  maintenance  philosophy,  requirements  for  maintenance 
functions  can  be  identified  and  formulated  into  Operator/Maintainer  Roles  and  Responsi¬ 
bilities.  Planned  Maintenance  (PM)  activities  such  as  checkout  (static  and  dynamic), 
cleaning,  removal  and  replacement,  etc.,  and  Corrective  Maintenance  (CM)  activities, 
such  as  fault  detection,  troubleshooting,  repair,  calibrations,  etc.,  can  be  identified  and 
classified  according  to  maintenance  functions. 

A  further  HFE  design  step  is  the  performance  of  a  Maintenance  Requirements 
analysis.  In  this  step  the  Human  Factors  Engineer  will  analyze  maintenance  requirements 
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to  electrical  components  using:  Failure  Modes  and  Effects  Analysis  (FMEA)  and  LOR 
data,  component  design  requirements  and  a  selected  maintenance  philosophy.  The 
analysis  also  entails  identifying  and  analyzing  information  requirements  for  each  mainte¬ 
nance  function,  identifying  and  analyzing  accessibility  requirements,  tool  and  test  set 
requirements  and  design  requirements.  This  step,  once  completed,  will  be  used  to  perform 
a  Maintenance  Task  Requirements  Analysis  and  to  develop  maintenance  man/machine 
design  concepts.  The  Maintenance  Task  Requirements  Analysis  requires  that  components 
and  maintenance  activities  be  identified  and  maintenance  functions  and  tasks  be  devel¬ 
oped.  Once  this  is  completed,  tasks  and  functions  are  analyzed  and  reduced  to  task 
elements. 

Tasks  and  task  elements  are  sequenced  and  analyzed  for  branching  steps.  Task 
requirements  are  identified  aiong  dimensions  such  as: 

•  Estimated  time  to  perform 

•  Information/communications 

•  Control  capabilities 

•  Display  capabilities 

•  Display  indication/information 

•  Equipment  design  features 

-  space  for  accessibility 

-  built  in  test  points 

-  tool  interfaces 

-  safety  provisions 

•  Skiil/knawledge 

•  Constraints  on  task  performance 

•  Frequency  of  task  performance 

•  Impact  of  error 

Task  criticality  and  priority  are  also  identified. 

Maintenance  Man-Machine  Interface  concepts  are  developed  by  analyzing  mainte¬ 
nance  activities  in  order  to  identify: 

•  Design  of  equipment  man/machine  interfaces  (controls/displays,  con¬ 
soles,  handles  and  handholds,  labels  and  markings,  packaging,  optics, 
etc.) 

•  Design  of  information  displays  (display  formats,  JPA's,  diagnostics, 
etc.) 

•  Maintenance  schedules  (Planned  Maintenance  (PM),  and  Corrective 
Maintenance  (CM)) 

Also  examined  are  maintenance  workspace  layout  and  arrangement  effects  by 
considering  such  maintenance  aspects  of  workspace  dimensions,  equipment  arrangement, 
maintenance  support  requirements,  equipment/compare  and  accessibility,  etc. 
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Maintenance  conditions  may  also  be  identified  aiong  such  dimensions  as  maintenance 
environment  (illumination,  temperature  and  humidity,  etc.)  and  operational  conditions 
such  as  states  of  readiness,  emergency,  bidckout,  etc.  Maintenance  personnel  require¬ 
ments  are  examined  along  issues  of  manning  (levels,  personnel  ratings)  procedures  (PM  and 
CM)  and  automation  levels  (automative  checkout,  manual  checkout,  etc.).  Maintenance 
training  requirements  need  to  be  identified  in  terms  of  skill  and  knowledge  required  at 
exit  of  training,  entry  skills  and  knowledge;  school  training,  JPA  or  OJT  trade-offs, 
training  media  and  methods  and  course  development  and  implementation. 

Operational  sequence  and  task  requirements  data  are  used  in  the  determination  of 
station  arrangements,  requirements  and  wo:  kspace  layouts. 

Preliminary  HFE  Design 

Spacial  station  arrangements  are  derived  in  order  to  facilitate  the  minimum 
required  traffic,  information  and  communication  flows.  Review  of  the  requirements 
analysis,  operational  sequence  analysis,  task  requirements  and  functional  allocations 
provide  the  necessary  information  to  generate  Station  Arrangement  Schemes.  Factors  to 
be  considered  are:  constraints  on  the  spacial  distribution  of  stations,  requirements  for 
traffic  flow,  requirements  for  information  flow,  and  required  communications  links 
between  stations.  A  link  analysis  is  often  performed  to  identify  and  analyze  required  links 
between  stations;  operational  sequence  data  provide  information  concerning  the  type  of 
links  (verbal,  electrical,  etc.)  between  stations  and  requirements  for  traffic  flow.  With 
these  data,  arrangement  schemes  can  be  generated. 

In  selecting  or  developing  various  station  layout  concepts,  a  list  of  items  to  be 
considered  is  established  (available  space,  communications  requirements,  etc.).  These 
factors  will  be  differentially  weighted  in  selecting  or  developing  arrangements. 

Station  workspace  layouts  and  arrangement  of  stations  are  typically  determined 
concurrently.  Generating  workspace  layouts  requires  that  the  following  requirements  are 
either  determined  or  identified  from  previous  analyses: 

•  Station  manning 

•  Control  functions 

•  Display  functions 

•  Communications  functions 

•  Equipment 

•  Environmental 

•  Visual  and  reach  envelopes 
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With  these  kinds  of  data  available,  individual  station  locations  and  orientations  can 
be  formulated  as  concepts.  In  order  to  evaluate  these  concepts,  criteria  are  developed. 
OSD  data,  task  requirements,  control  and  display  requirements,  etc.,  are  reviewed  for 
applicability  to  station  layouts,  and  criteria  are  formed  and  weighed.  Alternate  concepts 
are  then  selected  and  modified  until  an  optimum  workspace  layout  scheme  is  devised. 

At  this  point  in  the  HFE  Design  Process,  the  controls,  displays  and  communications 
requirements  for  each  station  are  identified.  Available  data  covers  the  allocation  of 
functions,  task  requirements  analysis  and  operational  sequence  analysis. 

For  those  operator  allocated  system  functions,  the  characteristics  of  control 
functions  must  be  identified,  enabling  individual  control  requirements  to  be  identified. 
For  each  task,  control  requirements  such  as  type  of  action  (continuous  or  discreet), 
criticality,  expected  frequency  of  use,  precision  requirements  for  continuous  controls  and 
required  feedback  information  are  identified. 

Requirements  for  individual  displays  are  similarly  identified,  per  task,  along  dimen¬ 
sions  such  as  the  information  to  be  displayed,  information  type  (continuous  or  discreet 
real  time  or  history,  status  or  performance),  criticiality  or  importance,  associated 
controls,  update  rate,  duration  of  information  presentation,  and  accuracy  requirements. 

Communications  requirements,  per  task,  are  identified  along  such  dimensions  as 
reporting  requirements  (frequency,  urgency,  system  status,  etc.),  standard  messages, 
information  dissemination  and  number  of  stations  reporting  and/or  receiving  information. 

In  recent  years,  computers  have  been  used  in  the  generating  of  displays,  aiding 
troubleshooting  and  logistics,  training  systems,  the  actual  control  of  system  functions  and 
the  storage,  retrieval,  analysis  and  dissemination  of  tactical  data.  Therefore,  formal  HFE 
analysis  of  man/computer  interfaces  is  required.  This  step,  within  the  HFE  design 
process,  identifies  interface  requirements  by  specific  functional  areas,  such  as:  monitor¬ 
ing,  verifying,  configuration  change/setup;  override;  programming;  debugging;  data  entry; 
mode  selecting;  data  maintenance  analysis  and  dissemination;  and  display  status  and 
projections. 

For  tasks  assigned  to  the  above  functional  areas,  identification  is  made  of: 

•  Required  information 

•  Required  control  actions 

•  Decisions 

•  Feedback 

•  Data  processing  information 
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•  Criticality 

•  Control  and  display  requirements 


Concepts  for  controls,  displays  and  communications  are  developed  by  first  identi¬ 
fying  constraints  such  as  cost,  conventions,  available  technology.  Panel  concepts  can  be 
generated  using  different  types  of  controls  and  displays,  communication  systems, 
man/machine  functional  allocations,  and  arrangements  and  are  based  in  part  on  human 
factors  design  criteria  such  as  importance,  sequence  of  use,  and  frequency  of  use.  For 
each  concept  generated,  link  analyses  and  error  likelihood  analyses  may  be  performed  to 
identify  potential  performance  problems  for  each  candidate  concept.  Console  and  panel 
arrangements  can  be  further  analyzed  by  the  establishment  oi  two  and/or  three 
dimensional  models  of  each  concept,  enabling  the  selection  of  feasible  panel  concepts  for 
trade-offs. 

In  developing  man/computer  interface  design  concepts,  identification  is  made  of 
constraints  such  as: 

•  Input/output  modes  and  requirements 

•  Message  formats 

•  Continuous  versus  call-up  display 

•  Override 

•  Display  symbology 

•  Program  selector 

Once  constraints  and  requirements  are  identified,  overall  man/computer  interface  con¬ 
cepts  can  be  generated,  and  feasible  concepts  selected  for  trade-offs. 

HFE  Training  System  Design 

Training  goals  are  identified  by  first  developing  behavioral  objectives  for  identified 
tasks  and  then  assigning  specific  tasks  to  pertinent  behavioral  objectives.  Performance 
conditions  and  standards  then  are  identified  for  each  objective. 

Training  media  and  methods  are  selected  by  identifying  course  requirements  such  as 
content,  phasing,  level  of  detail,  test  and  instructor  requirements.  After  reviewing 
constraints  and  factors  to  be  considered  in  selecting  a  training  method,  factors  can  then 
be  weighted  for  importance  and  a  training  method  selected. 

Media  selection  will  be  determined  by  the  following: 

•  Training  objective  priorities 

•  Requirements  for 

-  visual  presentations  of  information 
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-  motion  representation 

-  sound  representation 

-  branching 

-  prompting  and  cueing 

-  simulation 

Throughout  the  validation  phase,  systems  are  identified  for  advanced  development. 
As  these  systems  are  selected,  requests  for  proposals  are  prepared  for  engineering 
developments  of  subsystems  and  components.  Solicitations  for  limited  production  of  some 
items  may  also  be  advertised.  As  contractors  for  subsystem  and  component  engineering 
development  and  prototyping  are  let,  the  Test  and  Evaluation  plans  are  implemented. 

Input  to  subsystem  design  are  data  available  from  HFE  efforts  performed  to  date. 
Principal  inputs  are  the  results  of  maintenance  man/machine  interface  requirements  and 
control  console  concepts. 

HFE  Workspace  Concepts 

Man-machine  trade-off  criteria  for  controls,  displays  and  communications  typically 
are  functions  of  operability  dimensions,  and  equipment  reliability  and  cost  (life  cycle  and 
acquisition).  Operability  (or  human  performance)  dimensions  such  as:  error  likelihoods, 
response  and  performance  times,  operational  complexity,  training  time  and  requirements, 
skill  requirements,  workloads,  HFE  design  principles,  maintenance  requirements  and 
safety  are  formulated  into  man/machine  trade-off  criteria.  Requirements  for  additional 
data  such  as: 

•  Relative  man/machine  capabilities 

•  Relative  effectiveness  of  man  vs.  machine  operation 

•  Operational  procedures 

•  Workloads 

•  Skills 

are  first  identified,  and  a  study  test  plan  developed.  Requirements  for  mock-ups,  scene 
generators,  simulators,  measurement  apparatus,  etc.,  are  identified  prior  to  test  setup  and 
selection  of  test  subjects.  Studies  can  then  be  conducted,  the  data  analyzed  and 
interpreted,  and  results  inputted  to  trade-off  criteria.  With  these  criteria  available, 
trade-offs  are  performed  according  to  trade-off  method  selected,  criteria  weights,  ratings 
of  alternate  concepts  in  terms  of  conformity  with  criteria,  and  the  integration  of  weights 
and  rating  for  each  alternate  man/machine  concept. 


Console  concepts  are  generated  by  an  examination  of  control,  display,  communica¬ 
tions  and  man/computer  interface  concepts  as  well  as  operational  sequence  data,  JPA 


concepts,  man/machine  trade-offs  and  HFE  data  and  principles.  Specifically  addressed  is 
the  development  of  panel  specifications  (i.e.,  size,  shape,  orientation,  color),  control  and 
display  specifications  (types,  sizes,  shapes,  colors,  locations,  detents)  and  arrangements, 
communications  specifications  (messages,  modes,  etc.)  and  man/computer  interface 
specifications. 

Workspace  and  environmental  design  concepts  are  formulated  using  data  such  as: 

•  Equipment  requirements  at  each  station 

•  Environmental  affects  at  each  station 

-  illumination 

-  atmospheric  conditions 

-  noise  and  vibration  limits,  etc. 

•  Operational  requirements 

•  Maintenance 

Workspace  concepts  are  then  generated  according  to  controlling  constraints,  re¬ 
quirements,  and  environmental  effects. 

Milestone  II  Decision 

Prior  to  DSARC  II  (Milestone  II),  the  DCP  is  updated  to  contain  firm  program 
schedules,  cost  and  information  schedules.  Tne  DCP  is  forwarded  for  comment  to  the 
Defense  Acquisition  Executive  who  coordinates  the  review  activities  with  the  OSD  and 
OJCS.  The  DCP  and  comments  are  then  forwarded  to  DSARC  for  recommendations.  The 
DSARC  II  (Full-Scale  Engineering  Development)  recommendations  are  to  be  made  in 
accordance  with  the  following  Program  Issues  (as  per  DoD  5000.2): 

•  Mission  element  need  is  reaffirmed  and  the  threat  updated 

•  The  system  meets  mission  element  needs 

•  NATO  standardization  requirements  are  satisfied 

•  System  trade-offs  have  produced  the  optimum  balance  in  cost, 
performance  and  schedule 

•  Risks  have  been  identified  and  are  acceptable 

•  Planning  for  selection  of  major  subsystems  is  clearly  stated 

•  Testing  and  evaluations  have  been  completed  and  results  support 
recommendations 

•  The  TEMP  identifies  and  integrates  the  T&E  to  be  accomplished  prior 
to  DSARC  II  and  III 

Once  DSARC  has  reviewed  the  above,  recommendations  are  forwarded  to  the 
Secretary  of  Defense  for  approval  to  enter  the  full-scale  engineering  development  phase. 
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Requirements  to  Milestone  III 

2.2.3. 1  Full-Scale  Engineering  Development-Summary.  During  this  phase,  detailed 
ILS  specifications  are  generated,  the  Request  for  Proposals  for  the  weapon  system  is 
written,  full-scale  engineering  development  of  the  system  is  completed,  preparations  for 
production  are  made,  test  and  evaluation  is  continued  and  preparations  are  made  for  the 
Milestone  III  decision.  The  DSARC  process  continues  and  the  SECDEF  decides  either  to 
continue  full-scale  engineering  development,  cancel  the  program  or  enter  the  Production 
and  Deployment  Phase. 

The  primary  purpose  of  this  phase  is  "to  ensure  completion  of  sufficient  effort  to 
permit  a  confident  commitment  of  resources  required  for  quality  production" 
(SECNAVINST  5000.1).  This  phase  marks  the  beginning  of  the  preparation  of  contract  bid 
packages.  Drawings,  specifications  and  plans  are  collected  for  incorporation  in  the  RFP. 
A  Human  Factors  Engineering  section  of  the  RFP  may  include  requirements  for 
incorporating  and/or  developing  equipment  designs,  crew  complement  and  operator  roles. 
The  RFP  then  would  include  HFE  Data  Item  Descriptions  (DIDs)  which  state  HFE  which 
state  HFE  documentation  requirements  for  the  weapon  system  procurement. 

2.2. 3.2  Full  Scale  Engineering  Development  -  Detailed  Discussion.  According  to 
Geer  (1976),  the  decision  to  use  or  not  to  use  certain  DIDs  depends  upon  factors  such  as: 

•  The  extent  to  which  the  PM  wishes  to  ensure  performance  and 
documentation  of  HFE  analysis 

•  Cost  of  redundant/unnecessary  analysis  data 

•  Scheduling 

Geer  proposed  that  existing  HFE  DIDs  be  modified  and  proposes  three  relevant  to: 

•  Mission  Analysis  Report 

•  Functional  Allocation  Report 

•  Task  Analysis  Report 

After  contracts  have  been  awarded,  the  major  activities  include  combat  system 
design  and  integration  of  subsystems,  conduct  of  design  reviews,  equipment  prototyping, 
system  testing  and  preparation  for  DSARC  III. 

Involvement  on  the  part  of  the  Navy  is  limited  during  this  phase,  essentially  being 
limited  to  preparation  for,  and  conduct  of,  Test  and  Evaluation,  preparing  for  the 
impending  Milestone  III  decision  and  monitoring  contractor  activities. 
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HFE  Issues  During  Full  Scale  Engineering  Development 


Human  factors  requirements  turn  from  an  analytic  and  design  emphasis  towards 
highly  formal  design  evaluations,  and  design  criteria  and  procedures  development. 

Simulations  and  mock-up  evaluations  are  required  as  part  of  the  HE  effort  in  order 
to: 

•  Identify  design  problems  for  HFE  inputs  for  design  reviews 

•  Conceptualize,  document  and  verify  design  concepts 

•  Collect  data  for  workload,  timelines,  procedures,  etc. 

•  Identify  areas  where  redesign  may  be  required 

•  Develop  design  criteria 

The  level  of  sophistication  in  a  mock-up  evaluation  depends  largely  on  the 
complexity  of  the  system  and  the  number  and  magnitude  of  design  issues  to  be  examined. 
With  major  weapon  systems  development,  this  level  of  sophistication  is  typically  high. 
Full  sized,  functional  mock-ups,  or  if  sophisticated  enough,  simulators,  are  required. 
Detailed  evaluations  reveal: 

«  Operator/crew  workload 

•  Analysis  of  procedures 

•  Man-machine  interface  analysis,  etc. 

and  can  resolve  design  issues,  e.g.,  where  space  limitations  in  a  workspace  are  severe, 
simulations  will  indicate  design  criteria  along  operabiiity/maintainability  dimensions 
where  implementation  of  military  standards  (MIL-STD-1472,  for  example)  are  cleariy 
impossible.  Man-in-the-loop  simulations  are  also  implemented  to  evaluate  training 
systems  and  maintainability  design. 

Tne  major  considerations  of  mock-ups  and  simulations  is  that  of  fidelity.  The 
additional  experimental  control  that  is  afforded  by  mock-ups  and  simulators  is  offset 
somewhat  by  the  degree  to  which  the  simulation  can  approach  authenticity  of  the  actual 
equipment  and  environment. 

Based  essentially  on  simulations  and  mock-up  evaluations,  workload  limits  at  points 
within  the  mission  scenario  are  identified,  analyzed  and  inputted  to  detailed  design 
criteria  development. 
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3.0  HFE  TECHNOLOGY  REVIEW  AND  ASSESSMENT 


3.1  HFE  Technologies  Applicable  During  Feasibility  And  Program  Initiation  Phases 

Mission  Analysis 

Geer  (1976)  states  that  Mission  Analysis  "is  the  first  step  in  the  system  development 
required  for  the  establishment  of  human  factors  design  criteria". 

For  the  human  factors  specialist,  the  analysis  wiil  be  useful  in  subsequent  analytic 
techniques  such  as  analysis  of  similar  systems,  environmental  analysis,  functional  analysis, 
requirements  analysis,  operational  conditions,  and  functional  allocations.  Mission  analysis 
will  also  provide  the  analyst  with: 

•  An  understanding  of  the  mission 

•  Identification  of  mission  phases 

•  Mission  accuracy  requirements 

•  Mission  timing,  information,  urgency,  etc. 

Geer  points  out  factors  to  be  considered  in  establishing  Mission  Scenarios  and 
Mission  Profiles.  Selected  Mission  Scenario  factors  are  as  follows: 

•  Assumed  operational  factors 

•  System  and  subsystem  proposed  capabilities 

•  Postulations  of  geographic  positions 

•  Mission  starting  points  (time  and  location) 

•  Potential  deviations  from  established  mission  problem 

•  Development  of  alternate  profiles  based  on  threat  detectors 

•  Development  of  target  identification  techniques 

•  Target  engagement  techniques 

•  Evasive  maneuvers 

In  performing  mission  analysis,  the  above  factors  are  identified  (from  MENS, 
relative  force  levels,  etc.).  Mission  milestones  are  identified  (e.g.,  reach  cruise  altitude) 
and  can  be  used  to  segment  the  total  mission.  For  each  mission  segment,  identification  of 
factors  relevant  to  the  mission  segment  can  be  made. 

These  data,  once  gathered,  are  then  formulated  into  a  narrative  describing  the 
mission.  For  some  weapon  systems,  a  variety  of  missions  may  be  undertaken  (surveil¬ 
lance,  surveillance/attack,  attack)  and  a  mission  profile  and  scenario  for  each  will  be 
created  in  performing  Mission  Analysis. 
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Functional  Analysis 


Functional  Analysis  is  the  identification  and  analysis  of  broadly  defined  operations 
which  contribute  to  a  system  mission.  The  following  five  steps  are  sufficient  to  perform 
the  analysis: 

•  Identify  mission  and  operations,  and  for  each  identified,  determine 
any: 

-  constraints  on  function  performance 

-  requirements  for  function  performance 

•  Prioritize  missions  and  operations  by: 

-  frequency  of  occurrence 

-  performance  times 

-  criticality  to  total  mission 

-  difficulty 

•  Identify  major  mission  events 

•  Identify  system  functions  relevant  to  mission  phases  and  operations 

•  Analyze  functions 

-  functional  sequences 

-  functional  dependencies 

-  constituent  subfunctions 

Initially,  the  functional  analysis  is  a  simple  restatement  of  mission  analysis.  As 
subsystems  are  proposed  or  chosen  to  satisfy  mission  requirements,  the  analysis  is  iterated 
and  functions  are  determined  and  analyzed  at  greater  and  greater  levels  of  detail. 

Systems  Analysis  and  Integration  Model  (SAIM)  (Malone,  1967)  is  a  method  to  collect 
and  present  systems  function  and  requirements  information.  A  matrix  is  used  to  classify 
the  information  into  three  categories: 

•  Systems  determinants  -  nature  and  structure  of  the  system 

•  System  components  -  represents  systems  parts 

•  System  integrations  -  integrates  the  components  into  the  overall 
system 

Tne  matrix  is  generated  by  listing  in  both  the  rows  and  the  columns  the  identified 
system  determinants,  components  and  integrations  (Figure  <*).  The  appropriate  ceils  of 
the  matrix  are  checked  in  order  to  identify  where  column  entries  are  associated  with  row 
entries. 

A  method  for  analyzing  functions  is  the  Functional  Block  Diagram  (FBD)  (also  known 
as  Functional  Flow  Diagram).  This  is  performed  (typically)  by  the  following  steps. 

•  Formulate  functions  into  a  sequential  flow 

•  Develop  second  level  functions  based  on  top  level  functions,  mission 
requirements  and  functional  analysis. 
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FIGURE  4 

SYSTEMS  ANALYSIS  AND  INTEGRATION  MODEL  (SAIM) 


•  Iterate  until  a  levei  of  detail  is  evident  to  determine  how  a  function 
is  to  be  performed 

Two  examples  of  Functional  Block  Diagrams  are  presented  in  Figures  5  and  6. 

The  utility  of  the  Functional  Block  Diagram  for  the  human  engineer  is  to  provide  a 
detailed  sequence  of  mission/equipment  events,  a  sequential  outline  of  system  require¬ 
ments,  and  inputs  to  subsequent  HFE  activities  such  as: 

•  Functional  allocations 

•  Operational  sequence  analysis 

•  Task  analysis 

•  Timeline  analysis 

•  System  requirements  analysis 

Environmental  Analysis 

Based  on  mission  analysis  data,  an  Environmental  Analysis  can  be  performed  in  order 
to  identify  conditions  affecting  operational  issues  such  as  visibility,  communications, 
performance  and  safety.  Environmental  considerations  such  as: 

•  Time  of  day 

•  Glare 

•  Illumination  levels 

•  Atmospheric  conditions 

•  Weather  conditions 

•  Noise 

•  Vibration 

•  Acceleration  or  sea  state 

•  Shock 

•  Temperature 

for  each  mission  and  mission  phase  of  the  weapon  system  are  identified  and  become 
"design  to"  criteria  and  considerations  in  requirements  analysis. 

Requirements  Analysis 

A  Requirements  Analysis  is  applied  to  determine  information,  performance,  decision 
and  support  requirements  for  each  function  identified.  As  appiied,  the  analysis  usually 
entaiis  identifying  and  listing  requirements  for  each  function  identified.  Information 
requirements  include:  source,  accuracy  and  currentness.  Performance  requirements 
include:  accuracy,  time  limitations  to  perform/complete,  error  limits,  frequency  of 
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occurrence,  energy  expenditures.  Decision  requirements  such  as  options,  rules  and  error 
tolerances  are  also  identified.  These  data  serve  as  factors  to  be  considered  in  functional 
allocations,  task  anaiysis,  communication,  controi  and  display  requirements  identification. 
Mission  scenarios  and  profiles  are  the  principle  data  sources  for  the  analysis.  An  example 
of  a  performance  requirements  analysis  data  collection  form  is  presented  in  Figure  7. 

Functional  Allocation 

Within  the  Navy  (and  other  military  services),  manpower  issues  (personnel  types  and 
availabilities)  are  receiving  increasingly  greater  attention.  A  Navy  program,  "Military 
Manpower  Versus  Hardware  Procurement"  (HARDMAN),  has  recently  been  initiated  which 
will  attempt  to  integrate  manpower  requirements  throughout  the  weapon  system  acquisi¬ 
tion  process,  and  also  to  provide  hardware/manpower  tradeoff  guidance  as  an  integral 
aspect  of  system  design  (Boneau,  1979).  A  major  aspect  of  HARDMAN  implementation  is 
the  assessment  and  development  of  Human  Factors  Engineering,  Training  and  Manpower 
Planning  technologies.  In  fact,  a  driving  force  behind  HFE  technology  development 
(particularly,  evaluative  and  man-machine  tradeoff  (allocation)  technologies)  is  the 
HARDMAN  program,  and  that  HARDMAN  technology  development  emphasizes  evaluative 
and  tradeoff  technology  which  is  highly  consistent  with  A-109  and  current  acquisition 
strategy. 

With  the  availability  of  data  from  the  functional  analysis,  requirements  analysis  and 
analysis  of  similar  systems,  function  allocation  schemes  can  be  formulated  and  evaluated. 
The  principal  steps  taken  in  allocating  functions  are  to: 

•  Identify  constraints  on  allocation  (convention,  cost); 

•  Identify  or  estimate  level  of  system  automation; 

•  Identify  functions  best  performed  by  men  or  machines;  and 

•  For  functions  allocated  to  men,  establish  a  taxonomy  of  related 
functions. 

A  variety  of  techniques  are  available  to  perform  these  steps.  One  technique  called 
the  Evaluation  Matrix  (Geer  1976)  uses  sets  of  criteria  for  functional  allocation. 
Examples  are: 

•  Cost  (acquisition  and  life  cycle) 

•  Response  time 

•  Error  rate 

•  Reliability 

•  Survivability 
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FIGURE  7 

FORM  FOR  COLLECTING  PERFORMANCE  REQUIREMENTS 


Criteria  are  weighted  and  man  vs.  machine  allocations  are  scored  (on  a  scale)  for 
level  of  agreement  of  the  criteria.  An  example  of  an  evaluation  matrix  is  present  in 
Figure  S  (adapted  from  Geer,  1976). 

Another  technique  is  the  use  of  a  Relative  Capabilities  List  (Table  2  as  an  example). 
In  this  technique  a  function  to  be  allocated  is  compared  to  the  list  and  the  analyst 
determines  (more  or  less  subjectively)  the  mode  of  functional  performance  (man  or 
machine  or  both),  in  accordance  with  the  degree  to  which  a  function  is  most  suited  to  man 
or  machine  performance. 

The  Computer  Aided  Function-Allocation  Evaluation  System  (CAFES)  (Edwards,  et 
al,  1976,  Whitman,  1974,  Geer  1976,  Anderson,  1974)  provides  a  means  to  evaluate 
functional  allocation  schemes.  This  capability  is  provided  by  one  of  CAFES  five 
submodels,  the  Functional  Allocation  Model  (FAM).  The  remaining  CAFES  submodels  are: 

•  Data  Management  System  (DMS) 

•  Workload  Assessment  Model  (WAM) 

•  Computer  Aided  Crew  Station  Design  Model  (CAD) 

•  Crew  Station  Geometry  Evaluation  Model  (CGE) 

Since  CAFES  is  intended  to  be  a  comprehensive  tool  implemented  throughout  a 
systems  development  cycle,  CAD,  WAM  and  CGE  will  be  discussed  at  those  points  where 
their  application  is  tost  suitable,  leaving  the  present  discussion  to  DMS  and  FAM. 

DMS  provides  baseline  data  for  ail  other  CAFES  subsystems  and  has  three  purposes: 

•  Data  maintenance  (input,  editing,  storage) 

•  Interfaces  with  the  other  submodels  (in  terms  of  data  transfer) 

•  Output  data  direction 

DMS  is  comprised  of  four  different  modules: 

•  Editor  -  stores,  inputs  and  edits  data 

•  User  interface  -  accepts  directions  for  data  manipulation 

•  Executive  -  implements  other  submodels  and  prepares  data  files 

•  Report  generator  -  directs  output  as  specified  by  the  user 

DMS  is  essentially  the  medium  by  which  a  CAFES  user  implements  the  other 
submodels  and  maintains  a  system  data  base. 

FAM  is  designed  to: 

•  Identify  and  organize  system  functions 

•  Analyze  and  rank  order  various  functional  allocation  schemes 


TABLE  2 

RELATIVE  CAPABILITIES  LIST 


MAN 


Can  monitor  Low  pronaoility  events 
nor  feasible  for  automatic  systems 
because  at  numoer  at  events  pos¬ 
sible 

.Absolute  thresholds  at  sensitivity  are 
very  low  unoer  tavorabie  eondi- 
tions 

Can  detect  masked  signals  effectively 
in  overlapping  noise  spectra 

Able  to  acquire  and  report  information 
incidental  to  primary  activity 

Not  subject  to  jamming  by  ordinary 
methods 

Abie  to  recognize  and  use  information, 
redundancy  (pattern)  of  real  world 
to  simplify  camples  situations 

Reasonable  reliability  in  wnicn  the 
same  purpose  can  be  accomplished 
by  oifterent  approach  (corollary  of 
reprogramming  ability) 

Can  make  inductive  decisions  in  new 
situations;  can  generalize  from  few 
data 

Computation  weak  and  relatively  in¬ 
accurate;  optimal  game  theory 
strategy  cannot  be  routinely  ex¬ 
pected 

Channel  capacity  limited  to  relatively 
small  information  throughput  rates 

Can  handle  variety  of  transient  and 
some  permanent  overt oecs  without 
disruption 

Short  term  memory  relatively  poor 

Can  tolerate  only  relatively  low  im¬ 
posed  forces  and  generate  relative¬ 
ly  Low  forces  for  short  periocs 

Generally  poor  at  tracking  tnougn 
satisfactory  where  frequent  re pro¬ 
gramming  required;  can  change  to 
meet  situation,  is  best  at  position 
tracing  where  changes  are  under  3 
radians  per  second 

Performance  may  deteriorate  with 
time;  because  of  boredom,  fatigue, 
or  distraction!  usually  recovers 
with  rest 

Relatively  high  response  latency 

Relatively  inexpensive  for  available 
complexity  and  in  good  supoiy; 
must  :e  trained 

Light  in  weight,  small  in  size  for  func¬ 
tion  achieved:  power  requirement 
Less  than  100  watts 

Maintenance  may  require  ale  luooort 
system 

Nonexpendable;  interested  ji  rersonai  . 
survivals  emotional 


MACHINE 


Limited  program  complexity  and  alter¬ 
natives;  unexpected  events  cannot 
be  handled  adequately 

Generally  not  as  low  as  human  thresh¬ 
olds 

Poor  signal  detection  when  noise  spec¬ 
tra  overlap 

Discovery  and  selection  of  incidental 
intelligence  not  feasible  in  present 
designs 

Subject  to  disruption  by  interference 
and  noise 

Little  or  no  perceptual  constancy  or 
ability  to  recognize  similarity  of 
pattern  in  spartiai  or  temporal  do¬ 
main 

High  reliability  may  increase  cost  and 
complexity;  particularly  reliable  for 
routine  repetitive  functioning 

Virtually  no  caoacity  for  creative  or 
inducave  functions 

Can  be  programmed  to  use  optimum 
strategy  for  high-procabiiiry  situa¬ 
tions 

Channel  capacity  can  be  enlarged  as 
necessary  for  task 

Transient  and  permanent  overloacs  may 
lead  to  cisruption  of  system 

Short  term  memory  and  access  times 
excellent 

Can  withstand  very  large  forces  and 
generate  them  for  prolonged  periods 

Good  racking  characteristics  over 
Limixeo  requirements 


Senav.or  decrement  relatively  small 
with  time;  wear  maintenance  and 
product  quality  control  necessary 

.Arbitrarily  low  resoorae  latencies  pos¬ 
sible 

Complexity  anc  supply  limited  by  cost 
anc  times  performance  built  in 

Esuivaient  complexity  and  function 
would  require  radically  hesvter  ele¬ 
ments.  enormous  power  and  cooling 
resources 

Maintenance  orooiem  increases  dispro¬ 
portionately  with  complexity 

Expendable;  ron-personai:  will  perform 
wttneut  detraction 
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•  Analyze  and  output  data  for  the  preparation  of  Operational  Sequence 
Diagrams 

Input  data  for  using  FAM  suggest  its  complexity  and  comprehensiveness,  e.g.: 

•  Action  mode  (channel  activity,  tactile,  visual) 

•  Average  operator  reliability  for  a  nominal  task  time 

•  Earliest  task  start  time  during  a  mission 

•  Task  reexecution  time  for  interrupted  tasks 

•  Latest  tasks  start  time 

•  Machine  reliabilities 

•  Mission  objectives  (e.g.,  target  acquisition)  consisting  of  series  of 
dependent  tasks 

•  Mission  scenario  tasks  (time  based) 

•  Mission  start  time 

•  Mission  stop  time 

•  Mission  time 

•  Scenario  events 

•  Nominal  task  execution  times 

•  Number  of  task  repetitions 

•  Operator  reliability  (per  task) 

•  Task  priority  (task  interruptabiiity) 

•  Reliability  carve  data 

•  Task  reliability  weights  (relates  task  importance) 

•  RNO  -  Remaining  Number  of  Opportunities  to  execute  a  task  (as  a 
function  of  time  units  until  latest  start  time 

•  Pulse  constraints  (precedents  to  task  execution) 

•  Situations  during  mission  (equipment  malfunction,  etc.) 

•  Task  names 

•  Task  allocations 

•  Task  classification  (monitor,  operate,  etc.) 

•  Task  number  (for  user  identification) 

•  Task  load  rating  (sum  of  ratings  of  criticality,  interruptabiiity, 
reliability,  precision  and  concentration) 

•  Task  threshold  (maximum  task  load) 

•  Umbrella  tasks  (series  of  uninterruptabie  tasks) 

Task  and  mission  analysis  and  functional  analysis  are  relied  on  heavily  for  applica¬ 
tion  of  FAM  early  in  system  development.  Major  assumptions  are  required  (particularly 
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concerning  equipment  reliability)  for  very  early  FAM  implementation;  however,  as  system 
development  continues,  these  assumptions  become  fewer  and  more  valid. 

Two  procedures  of  FAM  are  the  Mission  Evaluator  and  the  Procedure  Generator. 
Tne  Mission  Evaluator  aspect  of  FAM  computes  mission  reliabilities  of  allocation 
schemes,  a  gross  workload  measurement  of  each  crewmember  and  man/machine  task 
reliabilities.  Principal  uses  of  these  aspect  data  are  to  select/modify  various  allocation 
schemes  and  to  assist  in  identifying  areas  where  specific  allocation  modifications  are 
required. 

The  Procedures  Generator  derives  data  for  the  development  of  operational  sequence 
diagrams  and  provides  procedure  statistics  for  allocation  schemes. 

The  Mission  Evaluator  procedure  is  as  follows: 

1.  Compute  individual  operator  and  machine  reliabilities 

2.  Compute  subsystem  reliability  (as  a  function  of  both  human  operator  and 
equipment  reliabilities) 

3.  Compound  task  reliabilities  to  determine  mission  success  probabilities 
(for  each  allocation  candidate) 

4.  Compute  mission  objectives  success 

The  human  reliability  computation  of  step  1,  above,  is  a  reliability  vs.  task 
execution  time  function,  i.e.,  the  human  reliability  for  a  given  task  cannot  be  determined 
until  a  task  execution  time  is  determined.  Simply  stated,  task  execution  time  is  a 
function  of:  (1)  nominal  execution  time  for  that  task;  (2)  perceived  task  load  (sum  of 
nominal  task  execution  times  for  remaining  operator  tasks  in  the  scenario);  and  (3)  elapsed 
mission  time.  Task  execution  times  are  altered  by  a  conversion  factor  in  accordance  with 
nominal  task  time  requirements  (to  complete  all  tasks  in  a  mission)  and  time  remaining  to 
complete  those  tasks.  Task  execution  time,  once  determined,  are  inputted  to  find  values 
of  operator  reliability  for  each  task.  Machine  reliabilities  are  used  as  input  by  the  user. 

Procedure  Generator  operation  is  as  follows: 

1.  Sort  tasks  according  to 

•  Must  tasks  (high  priority) 

•  Umbrella  tasks  (sequences) 

•  Regular  tasks 

2.  First  task  to  execute  is  determined  by: 

•  Results  of  step  L,  above 

•  Earliest  start  time 
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•  Task  dependencies 

3.  Clock  advances 

4.  Subsequent  tasks  are  selected  according  to: 

•  Earliest  start  time 

•  Latest  start  time 

•  Priorities,  umbrella  tasks 

5.  Tasks  are  selected  and  executed  until  either  all  tasks  have  been  completed,  or 

a  task  cannot  be  completed  in  available  time. 

FAM,  in  the  course  of  task  execution,  collects  and  stores  data  such  as:  task  start 
times,  task  scheduling,  task  interruptions,  task  time  percentages  (of  total  time)  and 
simultaneous  task  performances,  for  output. 

Specific  outputs  of  ,  AM  (Mission  Evaluator  and  Procedures  Generator)  are  as 
follows: 

•  Reliability  of  mission  (total  mission) 

•  Reliability  of  mission  objectives 

•  Crewmembers  workload  estimation 

•  Task  reliability  (redundant  man  and  machine  reliabilities) 

•  Percent  of  tasks  completed  and  interrupted 

•  Percent  of  mission  time  that  tasks  were  being  performed  simultane¬ 
ously 

Tosk  Analysis 

Task  analysis  refers  to  methods  used  to  specify  inputs,  behavioral  steps,  decisions 
and  actions  required  of  an  operator  to  effectively  perform  the  functions  which  have  been 
allocated  to  him.  Task  analysis  indicates  what  a  person  actually  'does'  rather  than  what 
he  is  'responsible'  for.  Meister  and  Rabideau  (1965)  have  described  task  analysis  as  ".  .  .a 
model  of  system  performance  in  terms  of  behavioral  elements  (perception,  decision 
making,  manipulation,  etc.)  in  relation  to  some  system  output". 

Although  emphasis  and  techniques  may  vary  between  human  engineers,  the  general 
philosophy  for  applying  task  analysis  is  relatively  consistent.  By  obtaining  an  accurate 
description  of  operator  tasks,  the  human  factors  specialist  can  begin  to  evaluate  the 
proposed  system  in  terms  of  the  appropriate  human  factors  criteria.  As  Kidd  and 
Van  Cott  (1963)  observe,  "The  objective  or  purpose  of  task  analysis  is  to  provide  the  basic 
'building  blocks'  for  the  rest  of  the  human  engineering  analysis". 
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In  system  development,  the  basic  process  of  task  analysis  is  one  of  inferring,  from 
the  actual  or  proposed  design  of  the  equipment,  what  tasks  and  sequences  of  tasks  will  be 
required  of  the  operator  when  the  system  is  completed.  In  addition  to  the  actual 
equipment  to  be  employed,  the  human  engineer  must  be  cognizant  of  such  factors  as 
safety  and  maintenance  requirements  which  will  impact  operator  performance.  In 
conducting  the  analysis,  the  human  factors  specialist  must  first  collect  and  organize  all 
available  information  regarding  the  system  especially  for  those  functions  allocated  to 
human  performance.  Generally,  this  is  accomplished  through  the  use  of  a  predesigned 
task  analysis  form,  such  as  those  depicted  in  Figures  9  to  15.  Figure  9  presents  the 
format  used  by  ESSEX  Corporation  and  displays  the  types  of  information  that  were 
gathered  in  the  analysis  of  an  actual  system. 

In  developing  a  task  analysis,  the  following  types  of  information  are  collected  and 
recorded  in  the  appropriate  row  or  column  on  the  data  form. 

•  STATION  -  records  the  name  of  the  station  or  operator  position  at 
which  the  task  is  to  be  accomplished. 

•  DUTY  -  identifies  the  system  process  of  which  the  task  is  a  com¬ 
ponent. 

•  TASK  -  identifies  the  specific  task  which  is  to  be  accomplished. 
Entries  into  this  column  are  ordered  sequentially  to  assist  the  analyst 
in  developing  time  lines,  operational  sequence  diagrams,  etc. 

•  ACTIVITY  -  describes  what  the  operator  must  do  to  complete  the 
task.  The  description  should  contain  an  action  verb  which  adequately 
describes  the  operator's  response  (e.g.,  monitors,  actuates,  signals, 
etc.). 

•  EST.  TIME  (MIN.)  -  presents  the  amount  of  time  estimated  for  the 
operator  to  complete  the  necessary  activity.  These  data  are  useful  in 
evaluating  the  ability  of  the  system  to  operate  within  established 
time  constraints. 

•  FREQUENCY  -  of  the  activity. 

•  INFORMATION/COMMUNICATION  REQUIREMENTS  -  Under  this 
heading,  the  human  engineer  describes  the  type  of  information  the 
operator  needs  to  perform  the  task,  or  the  communication  require¬ 
ments  of  the  task.  The  stimulus  may  be  an  out-of-tolerance  display 
indication,  an  indication  that  maintenance  is  required,  a  signal  from 
another  operator,  or  some  similar  input  that  indicates  the  need  to 
respond. 

•  CONTROL  -  In  this  column,  the  analyst  enters  the  name  or  descrip¬ 
tion  of  the  control  used  for  the  activity. 

•  DISPLAY  -  In  this  column,  the  analyst  describes  the  display  used  by 
the  operator  to  perform  the  activity. 

•  INDICATION  -  Under  this  heading,  the  human  engineer  describes  the 
type  or  source  of  feedback  available  to  the  operator  which  indicates 
that  the  necessary  system  response  has  occurred. 
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FIGURE  9 

ESSEX  CORPORATION  STANDARD  TASK  ANALYSIS  FORMAT 


12-5  block  Under  wheel  Decision  to  lest  «de<|uucy  o(  block.  Emergency  broke  Release  Car  dues  nol  roll  dov/n  hill 

(Concepts  12,  19) 


UtOSS  I  ASK  AIIALYSIS 


FIGURE  12 

TWO-LEVEL  TASK  ANALYSIS 


I  ASK:  (2)  I  Control  |el  engine  opera! ion 


FIGURE  13 

REPRESENTATIVE  TASK  ANALYSIS 


FIGURE  14 

BEHAVIORAL  ANALYSIS  OF  TASKS  WORKSHEET 


TEXT  GRAPHICS 


RESPONSE 

CUE 

CONTEXT 

FOCUS 

Access 

1. 

Pull  hood  latch  to 
left  and  hold 

Hood  latch 

View  of  vehicle  front 

Hood  latch 

2. 

Lift  hood,  then 
release  latch 

Hood 

View  of  hood  raised, 
support  bar  straight 

Hood 

3. 

Pull  support  bar  to 
front 

Support  bar 

View  showing  support 
bar  in  final  position 

Support  bar 

Remove  Air  Cleaner  Base 

1. 

Using  blade  screw 
driver  loosen  lower 
clamp  screw 

Clamp  screw 

View  of  right  side  of 
engine 

Inset  of  clamp  screw 

2. 

Loosen  clamp 

Clamp 

View  showing  clamp 
loose 

Clamp 

3. 

Slide  off  air  cleaner 
hose 

Air  cleaner  hose 

View  showing  hose 
removed 

Air  cleaner  hose 

Remove  Vacuum  Line 

1. 

Using _ wrench 

unscrew  vacuum  line 
nut 

Vacuum  line  nut 

View  of  right  side  of 
engine 

Inset  of  nut 

2. 

Pull  out  vacuum  line 

Vacuum  line 

View  showing  vacuum 
line  disconnected 

Vacuum  line 

Remove  Throttle  Linkage 

1. 

Using _ wrench 

unscrew  nut 

Throttle  link  nut 

View  of  right  side  of 
engine 

Inset  of  nut 

2. 

Puli  linkage  out 

Linkage 

View  showing  linkage 
disconnected 

Linkage 
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FIGURE  15 

MAINTENANCE  TASK  ANALYSIS 


•  SKILLS/KNOWLEDGE  -  The  analyst  uses  this  column  to  describe 
skills  and/or  knowledge  required  by  the  operator  to  complete  the 
task. 

•  POTENTIAL  ERRORS/CQNSTRAINT5  -  In  this  column,  the  analyst 
lists  probable  sources  of  error  based  on  the  type  of  response  required 
of  the  operator  and  characteristics  of  the  equipment  used. 

•  ERROR  IMPACT  -  Under  this  heading,  the  human  factors  specialist 
describes  the  effect  the  various  possible  errors  will  have  on  mission 
effectiveness. 

The  quality  of  information  available  to  the  human  engineer  and  the  purpose  of  the 
analysis  will  determine  the  detail  to  which  the  analyst  describes  the  various  components 
of  the  task. 

The  data  gathered  in  the  course  of  a  task  analysis  have  valuable  applications 
throughout  the  human  factors  design  process.  Examples  of  these  applications  are 
presented  below. 

•  Manning  Requirements  -  By  examining  the  types  of  tasks  to  be 
accomplished  and  tfie  skill-knowledge  requirements  of  the  operator, 
the  human  engineer  may  determine  the  number  and  types  of  person¬ 
nel  needed  to  accomplish  the  mission. 

•  Training  -  Task  analysis  provides  the  human  factors  specialist  with 
the  information  necessary  to  specify  the  level  of  competency  re¬ 
quired  of  an  operator  to  effectively  perform  at  the  designated 
workstation. 

•  Workstation  Design  -  The  control  and  display  information  provided  by 
the  task  analysis  will  assist  the  human  engineer  in  describing  the 
types  of  equipment  necessary  for  the  workstation.  The  sequential 
ordering  of  the  tasks  and  the  frequency  of  the  activities  will  aid  the 
analyst  in  determining  the  optimal  configuration  of  the  equipment  at 
the  station. 

•  Communication  Requirements  -  Data  generated  by  the  task  analysis 
will  help  the  analyst  determine  the  communication  links  (source  and 
content)  which  will  be  required  by  the  station  operator. 

•  Maintenance  Requirements  -  The  value  of  task  analysis  to  main¬ 
tenance  requirements  is  two-fold.  First,  the  analysis  provides 
information  regarding  the  precision  required  of  the  equipment.  Sec¬ 
ond,  a  separate  task  analysis  can  be  performed  to  describe  the 
maintenance  process  itself. 

•  Job  Performance  Aids  -  To  develop  effective  job  performance  aids, 
the  human  engineer  requires  detailed  descriptions  of  what  tasks  the 
operator  must  perform  and  what  skills  and  information  are  required 
to  perform  them.  A  well  developed  task  analysis  may  be  used  as  a 
step-by-step  introduction  to  the  system,  or  as  a  source  for  opera¬ 
tional  and  training  manuals. 

•  Test  and  Evaluation  -  The  descriptions  of  operator  performance  in¬ 
herent  in  task  analysis  can  be  used  to  generate  performance  criteria 


with  which  to  evaluate  the  effectiveness  of  a  system  and/or  its 
operators. 


The  following  are  examples  of  task  analysis  methods,  varying  as  to  terminologies, 
formats,  taxonomies  and  types  of  behavioral  features  and  emphasis. 

A  method  developed  by  Chenzoff  and  Foiley  (1965)  places  tasks  in  behavioral  classes 
to  be  used  in  training  design  decisions.  The  activities  (components)  are  coded  according 
to  their  main  tasks.  The  columns  list  the  following: 

•  Number  of  task  code  (activity) 

•  Person  performing  the  activity 

•  Type  of  activity 

-  procedural 

-  monitoring 

-  perceptual  motor 

-  communication 

-  decision  making 

•  Sequence  of  the  activity 

-  fixed  (F) 

-  variable  (V) 

•  Ratings 

-  Q-not  essential 

1 - necessary  but  not  demanding 

2 - critical 

•  Coordination 

-  number  of  persons  needed  to  perform  the  task  (alone  or  a 
team) 

1  =  1  man 
2=2  men 
3=3  or  more 

•  Specialized  Behavior  -  Amount  of  training  required  to  progress  from 
entry-level  behavior  to  exit-level  behavior 

Q=not  related  to  previous  experience 
1= readily  learned 

•  Difficulty 

•  Dynamic  Condition 

•  Remarks 

Review  of  the  data  gathered  can  thus  enable  the  analyst  to  set  up  the  functional 
requirements  of  the  training  system. 

Miller's  (1963)  format  is  that  of  a  matrix  with  tasks  assigned  along  one  coordinate 
representing  requirements  for  different  aspects  of  training  and  along  a  second  coordinate 
listing  the  different  phases  of  work.  The  ceils  within  the  matrix  are  coded  to  indicate  the 
task  and  training  equipment  needed  for  the  training  aids.  These  codes  are  formed  from 
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separate  activity  descriptions  (i.e.,  time  demands,  perceptuai  difficulties,  feedback,  etc.). 
Miller  also  characterizes  the  different  trainers  avaiiabie  as  to  "principal  feature,  critical 
factor,  training  value  and  limitations"  and  human  engineering  aspects. 

The  steps  are: 

1.  Listing  of  tasks  at  a  broad  level 

2.  Grouping  tasks  as  to  time  or  kind  or  both 

3.  Reviewing  activities  in  each  task  in  light  of  task  groups 

4.  Coding  as  to  use  in  different  stages  of  training 

In  addition,  there  is  a  task-time  chart,  a  list  of  conditions  under  which  work  is 
performed  and  a  catalogue  of  activities  performed  in  the  task  (perceptual  difficulties, 
decision  making,  time  demands,  equipment  malfunction  and  correction,  attention,  short¬ 
term  recall,  long-term  knowledge  for  contingencies,  correctness  of  response  feedback  and 
displays,  and  controls). 

Miller's  instructions  for  performing  the  task  analysis  are: 

1.  Prepare  statement  of  requirements 

a.  list  types  of  missions  (if  tasks  vary  from  one  mission  to  another) 

b.  list  tasks 

c.  prepare  block  diagram  of  tasks  as  they  occur  in  the  work  cycle 

d.  describe  the  conditions  under  which  the  tasks  are  performed 
(including  the  "signs  whereby  the  operator  recognizes  the  need  for 
performing  the  task") 

e.  describe  the  activities  in  the  task  (including  information  regarding 
time  demands,  etc.) 

2.  Prepare  table  of  tasks  to  be  taught  and  types  of  trainers 

The  basic  element  of  this  type  of  task  analysis  is  the  "classification  of  training 
devices  on  a  kind  of  habit  or  skill  provided  the  trainee,  rather  than  the  subject  matter 
taught". 

De  Maree's  (1961)  form  of  task  analysis  utilizes  a  descriptive  list  of  tasks  presented 
in  tabular  form  and  is  based  on  "a  list  of  behavior  with  several  broad  stages  of  training 
creating  groups  of  tasks  called  'training  functions' ".  Each  of  these  functions  is 
categorized  according  to  ten  "Training  Equipment  Effectiveness  Characteristics".  The 
object  is  to  scale  each  of  these  ten  in  order  to  define  the  tasks  as  to  their  specifics  and  as 
to  trainees  and  context  effectiveness  characteristics. 

Steps  taken  on  Training  Equipment  Requirements  Data  Sheet  are: 

1.  Task  identification  (from  "Quantitative  and  Qualitative  Personnel  Re¬ 
quirements  Information  Report") 

2.  Differentiation  of  training  functions: 


a.  knowledge 

b.  skills  and  task  components 

c.  whole  task  performance 

d.  integrated  task  performances 

e.  familiarization  trainers 

f.  instructed-response  trainers 

g.  automated  skill  trainers 

3.  Proficiency  levels  tabulated  based  on  extent  of  supervision,  etc. 

4.  Effectiveness  characteristics  coded  by  degree  of  complexity 

5.  Degree  of  utilization  estimated 

De  Maree's  method  is  similar  to  Miller's  with  the  addition  of  means  for  eliciting 
information  regarding  training  needs,  level  of  proficiency  and  the  trainee's  inputs.  It  is, 
therefore,  a  more  complete  and  explicit  form  of  task  analysis. 

According  to  Sherrill  (1976),  the  purpose  of  task  analysis  is  the  design  of  criterion 
tests,  since  good  performance  tests  must  be  designed  before  training  can  be  designed. 

The  size  of  the  task,  the  boundaries  of  the  task  and  the  potential  sources  of  variance 
are  the  key  concepts  used  by  Sherrill. 

•  Concept  1  -  Size  of  task.  A  task  can  be  a  part  of  a  larger  task,  and 
tasks  should  be  set  up  on  a  continuum — from  the  smallest  to  the  most 
complex. 

•  Concept  2  -  Boundaries  of  tasks.  This  is  basically  the  information 
gathered  from  a  task  analysis.  The  categories  of  information  are: 

-  the  initiating  cues 

-  the  terminating  cues 

-  the  givens 

All  three  elements  must  be  identified  when  performing  a  task 
analysis  to  decide  what  type  of  training  criteria  are  needed. 

•  Concept  3  -  Identification  of  potential  sources  of  variance.  During 
task  analysis  a  systematic  laying  out  of  the  task  should  bring  out  the 
possibilities  of  misses,  incorrect  performance  or  sources  of  variance. 
Different  levels  of  specificity,  from  macro  to  micro,  enable  the 
analyst  to  back-track  when  he  observes  a  problem  not  recorded 
before.  A  type  of  task  analysis  used  would  be  the  logic  tree.  Also 
used  is  an  outline  form  which  depicts  a  straightforward,  stepwise 
analysis  of  a  task— step  one,  two,  three  (see  page  3).  The  branching 
form  is  frequently  encountered  in  maintenance  manuals.  The  vari¬ 
able  format  is  used  in  tasks  in  personnel  work— tasks  which  have 
many  randomly  occurring  cues  and  many  different  outcomes  from  the 
different  sets  used. 

In  determining  which  of  these  forms  to  use,  Sherrill  suggests  examining  the  task 
description.  If  compound  or  complex  sentences  are  used  and  words  such  as  "except", 
"unless",  "but",  and  "when"  appear,  the  variable  form  should  be  used  to  analyze  the  task. 

All  three  concepts — outline,  branching  and  variable— are  used  in  identifying  the 
outcome  of  the  task  (the  criteria)  before  the  inputs  (learning  materials)  are  developed. 
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As  part  of  the  development  cycle  for  FP3PA  (fully  proceduralized  JPAs)  Shriver 
(1975)  proposes  BAT  (Behavioral  Analysis  of  Tasks)  which  "would  surface  most  of  the 
important  environment  cues  that  would  be  missed  by  a  conventional  'hands  on'  analysis, 
cues  that  are  necessary  for  highly  effective  FP3PA". 

The  basic  BAT  format  involves  "Cues"  and  "Responses".  The  analyst  must  highlight 
each  individual  cue  which  elicits  a  response  from  the  individual.  For  example,  if  a 
maintenance  man  is  to  open  an  access  panel,  there  is  no  overall  cue  for  this  response  since 
all  he  sees  is  the  panel.  However,  the  next  cue  would  be  the  screws  holding  the  panel. 
His  response  to  these  cues  wouid  be  to  put  up  a  screwdriver  and  turn.  This  goes  into  the 
Response  column.  The  next  step  involves  a  CUE  entry  of  turning  a  handle  to  open  the 
panel.  If  the  cue  is  anything  but  a  simple  action,  an  additional  cue  must  be  sought  by  the 
analyst  and  recorded. 

Graphics  are  produced  from  the  cues  to  produce  a  diagram  which  the  maintenance 
man  can  compare  with  the  real  equipment,  such  as  a  picture  of  the  inside  of  the  panel 
after  the  cue  of  opening. 

The  BAT  is  constructed  step-by-step  and  with  full  detail.  It  also  includes  error 
information,  or  what  will  happen  if  a  step  is  performed  incorrectly.  Each  step  in  the  task 
analysis  includes  a  verbal  description  and  a  diagram. 

Shriver  acknowledges  that  the  construction  of  a  BAT  requires  expensive,  highly 
skilled  and  tedious  work  on  the  part  of  the  analyst.  He  feels,  however,  that  its  use  in  the 
FP3PA  development  cycle  will  lead  to  a  quality  result  on  minimum  cost. 


Operational  Sequence  Analysis 


Along  with  application  of  Task  Analysis,  development  of  the  Operational  Sequence 
Diagram  (OSD)  can  begin.  The  OSD  (Figures  16  and  17)  is  a  graphic  representation  of  the 
operation  of  a  system  which  shows:  operational  links,  communications  networks  and  links, 
the  phasing  and  syncronizing  of  events,  functional  allocations,  man/machine  interactions, 
decision  points  and  operational  task  dependencies.  The  technique  is  highly  valuable  to  the 
HF  engineer.  The  OSD  can  be  used  to: 


•  Develop  operational  procedures 

•  Evaluate  man/machine  interfaces 

•  Evaluate  functional  allocations 

•  Identify  critical  mission  areas 

•  Identify  task  over-  and  underload  areas  for  given  operators  during 
given  mission  phases 
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•  Identify  critical  decision/action  points 

•  Develop  workspace  design  and  evaluation  criteria 

•  Identify  areas  of  high  error  likelihood 

In  short,  the  OSD  should  (at  various  times  in  the  systems  development)  provide 
information  sufficient  to  aid  in  the  design,  evaluation  and  documentation  of  the  system. 
The  reason  that  the  technique  is  so  valuable  lies  in  the  fact  that  in  order  to  establish  a 
good  OSD,  a  great  deal  of  information  must  be  gathered  and  analyzed  before  the  diagram 
can  be  developed.  Data  to  develop  the  OSD  are  gathered  from  Functional  Analysis 
(specifically  the  Functional  Block  Diagram),  Task  Analysis  (for  task  sequencing,  identifi¬ 
cation  of  task  type,  duration,  etc.),  Functional  Allocations,  analysis  of  similar  systems  and 
requirements  analysis. 

The  OSD  is  typically  an  iterative  technique,  beginning  with  a  description  of  more  or 
less  theoretical  system.  As  development  of  the  system  and  implementation  of  HF 
techniques  continues  (subsystems  identification  of  functional  reallocations,  workstation 
designs,  equipment  designs,  so  on)  the  data  are  incorporated  into  the  OSD  such  that  it  is 
maintained  to  reflect  current  system  configuration.  Use  of  the  OSD  itself  (as  an 
evaluation  tool)  will  suggest  design  and  operational  changes,  calling  for  an  updated  version 
of  the  tool. 

The  fact  that  the  OSD  requires  a  good  deal  of  input  information  and  is  made  to  be 
current  with  progressively  greater  system  detail  makes  it  an  expensive  and  time 
consuming  technique.  The  Automated-OSD  (A-OSD)  has  been  developed  to  help  alleviate 
these  problems  (Lahey,  1970,  Larson  and  Willis,  1970).  The  aid  (which  emphasizes  the 
usefulness  of  OSD  as  a  timeline),  uses  a  character  printer  to  make  OSDs.  Lines  are 
represented  by  strings  of  dots  (periods)  and  rather  than  the  use  of  symbols,  characters  of 
the  alphabet  are  used  to  indicate  types  of  actions: 

•  Decision 

•  Inspection 

•  Operation 

•  Recall/retrieve  (information) 

•  Store 

•  Transmit 

and  mode  of  action: 

•  Bodily 

•  Cognitive 


3-32 


•  Electrical 

•  Mechanical 

•  Auditory 

•  Tactile 

•  Visual 

Descriptions  of  each  entry  in  the  "diagram"  are  printed  to  the  right  of  the  action 
(e.g.,  "monitors  screen"). 

Changes  and  additions  to  the  OSD  are  entered  mechanically  and  a  new  OSD  is  then 
printed.  The  developers  state  that  lines  can  easily  be  drawn  manually  and  that  the  use  of 
characters  as  descriptors  are  easily  learned. 

It  seems  evident  that  an  interactive  program  for  the  development  of  an  OSD  would 
be  of  great  benefit  for  the  updating  of  the  diagrams,  particularly  with  larger,  more 
complex  OSDs. 

Other  types  of  Operation  Sequence  Diagrams  are  the  Spaciai  OSD  (S-OSD)  and  the 
Task  Analysis  -  OSD  (TA/OSD).  The  first  type,  the  S-OSD  (see  example  in  Figure  18),  is 
very  much  like  a  link  analysis.  It  represents  a  panel  concept,  which  incorporates  the 
sequences  of  events  of  an  operator  as  he  interacts  with  the  panel. 

The  TA/OSD  is  a  marriage  of  Task  Analysis  and  Operational  Sequence  Diagramming. 
An  example  of  TA/OSD  is  presented  in  Figure  19.  The  technique,  as  can  be  seen  by  the 
figure,  provides  more  detailed  task  descriptions  than  previously  described  OSDs.  This 
technique,  although  more  complex  and  less  suited  (perhaps)  for  computerization  (using  a 
graphics  terminal),  provides  more  information  that  is  readily  available  to  the  HF  analyst 
than  task  analysis  or  OSDs  alone. 

In  preparing  the  TA/OSD,  the  minimum  data  requirements  are  task  sequences, 
functional  allocations  and  task  requirements  data.  The  diagram  and  task  analysis  may 
proceed  in  parallel,  i.e.,  analyze  tasks,  incorporate  into  OSD,  analyze  OSD,  input  to  task 
analysis. 

A  useful  tool  for  assisting  Task  Analysis  and  OSD  development  is  the  Decision- 
Action  Diagram.  The  purposes  of  the  tool  are  to  identify,  analyze  and  graphically 
represent  decision  points  in  terms  of:  (1)  decisions  to  be  made;  (2)  options;  (3)  rates;  and 
(4)  subsequent  actions  taken  as  a  resuit  of  specific  decisions.  Analysis  of  the  diagram  will 
reveal  common  actions  performed  at  different  decision  points,  common  decisions  and 
series  of  unique  decision  sets,  and  decision  set  outputs.  The  analysis  can  also  help  to 
identify  critical  decision  points,  required  information  and  potential  effects  of  erroneous 
decisions. 
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BEGIN 

2X11  SEQUENCE 


1.  Observe  System  Requirement 

2.  Observe  Input  Data  Requirement 


3.  Determine  Input  Channel 

4.  Make  Input 

5.  Observe  Feedback  Indication 

6.  Agree  with  Required  Input? 

7.  Seiect  Aajustment  Mode 

8.  Perform  Adjustment 

FIGURE  IS 
SPACIAL  OSD 
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The  method  of  diagram  development  is  tedious  but  relatively  straightforward. 
Requirements  analysis  is  reviewed  to  assist  in  identifying  decision  points  and  options.  At 
each  decision  point  and  for  each  decision  option,  subsequent  activities  and/or  decisions 
are  diagrammed,  and  subsequent  decisions  and  actions  are  likewise  diagrammed  for  each 
potential  output  of  previous  decisions  or  actions. 

An  example  of  a  Decision-Action  Diagram  for  a  relatively  simple  operation  is  shown 
in  Figure  20. 

3.2  Technologies  Applicable  During  Validation  Phase 

The  major  efforts  of  the  HF  engineer  during  this  phase  are  to  develop  concepts 
towards: 

•  Workspace  design  (including  controls,  displays,  consoles,  panels,  envi¬ 
ronment,  communications  and  so  forth) 

•  Maintenance  designs 

•  Training  systems  designs 

and  to  evaluate  these  concepts  by  application  of  evaluative  technologies  and  formal  HFE 
Test  and  Evaluation. 

Of  the  technologies  surveyed  which  are  applicable  to  this  phase  most  fail  into  two 
categories:  diagnostic  design  and/or  diagnostic/evaluative  (typically  for  design  trade¬ 
offs).  Technologies  for  design  are  scarce;  typically,  design  concepts  are  formulated  from: 

•  The  HFE  data  bank  for  the  system  (OSDs,  task  analyses,  etc.) 

•  Identification  of  design  constraints  and  requirements  (cost,  conven¬ 
tion) 

•  Application  of  human  factor  design  principals 

•  Design  standards  (MIL  STD  1472B,  for  example) 

The  basic  procedure  for  the  HF  engineer  during  this  phase  are: 

1.  Requirements/constraints  identification 

2.  Evaluation  and  trade-offs 

3.  Design  modifications  and  trade-offs 

4.  Iterations  of  steps  above  as  required 

Controi/Display  Selection 

HFE  Principles  and  Criteria  can  aid  in  the  selection  and/or  design  of  controls  and 
displays.  The  Human  Engineering  Guide  to  Equipment  Design  (Van  Cott  and  Kinkade, 
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1972)  provides  general  guidelines  in  terms  of  control  and  display  selection.  Specific 
information  requirements  for  control  selection  are:  function,  task  requirements  (pre¬ 
cision,  speed),  information  requirements,  workspace  requirements  (layouts,  workspace 
availability),  and  consequences  of  control  error. 

General  principles  are  also  provided,  e.g.: 

•  Controls  should  not  overburden  any  particular  limb 

•  Continuous  controls  should  be  used  for  requirements  where  con¬ 
tinuous  settings  exist 

•  Discrete  controls  should  be  selected  for  discrete  system  settings 

•  Functionally  related  controls  should  be  combined 

Criteria  for  Controls  and  Displays  are  available  in  MIL-STD-1472,  NATO  Agree¬ 
ments,  etc.  MIL-STD-1472  provides  criteria  regarding: 

•  Integration 

•  Sizes/shape 

•  Colors 

•  Labelling 

•  Reach  distances 

•  Number  of  discrete  positions,  etc. 

for  various  types  of  Controls  and  Displays  (cranks,  CRTs,  legend  lights,  etc.). 

AIR  Data  Store 

The  American  Institute  for  Research  Data  Store  (AIR  Data  Store)  (Meister,  1971) 
was  developed  to  aid  in  the  selection  of  controls  and  displays  for  workspaces  via 
predictions  of  performance  time  and  reliability  of  performance.  The  AIR  Data  Store  is  in 
part  a  compilation  of  control  and  display  types  and  predicted  execution  times  and 
reliabilities  for  each. 

The  procedure  for  application  of  the  data  store  is  as  follows.  For  each  required 
control  and  display,  a  type  is  selected  (pushbutton,  toggle  switch,  guage,  so  on)  and  the 
data  store  is  referenced  for  predicted  mean  execution  time  and  probability  of  success  in 
terms  of  operation.  Execution  times  are  added,  yielding  total  time  (for  operation)  and  the 
probabilities  compounded.  The  goal  is  to  minimize  total  time  and  maximize  reliability  via 
control/display  type  selection. 

While  the  AIR  Data  Store  does  not  contend  with  such  issues  as  decision  reliability 
(e.g.,  an  operator's  selection  of  the  appropriate  control),  it  may  be  valuable  as  a  trade-off 
tool  or  an  equipment  selection  tool. 


J 
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Link  Analysis 

Link  analysis  is  a  method  which  aids  in  the  development  of  man  and  machine 
arrangements.  The  purpose  of  link  analysis  is  to  aid  in  minimizing,  for  any  system  or 
workspace,  considerations  such  as  traffic  flow,  observational  distances,  and  information 
flow  distances. 

Data  requirements  for  the  analysis  are  as  follows: 

•  Information  flow  requirements 

•  Information  flow  medium  (auditory,  visual,  ambulatory,  electronic) 

•  Station  requirements  (number  of  stations,  number  of  operators, 
functioned  allocations) 

•  Special  allocation  constraints 

According  to  Thompson  (1972),  link  analysis  is  applied  in  nine  discrete  steps,  as 
follows: 

•  Identify  (by  a  circle,  on  a  piece  of  paper)  each  required  operation 

•  Identify  (by  a  square)  equipment  requirements 

•  Identify  links  (lines)  between  appropriate  men 

•  Identify  links  between  men  and  machines 

•  Simplify  the  arrangement  by  reducing  the  number  of  crossing  links 

•  evaluate  each  link  for  importance  and  frequency  of  use 

•  Redraw  the  diagram  reducing  link  length  and  number  of  crossing  links 
in  accordance  with  step  6 

•  Fit  the  diagram  into  the  available  work  area  (by  redrawing  if 
necessary),  or  design  work  area  around  link  diagram 

•  Confirm  the  link  diagram  by  drawing  to  scale  equipment  and  ma¬ 
chines. 

The  link  analysis  (Figure  21)  can  be  applied  at  a  variety  of  levels,  e.g.,  a  CIC 
workspace,  or  at  a  total  systems  level,  such  as  a  ship  or  aircraft. 

Correlation  Matrix 

A  tool  to  expand  the  power  and  utility  of  the  link  analysis  is  the  Correlation  Matrix 
(Geer,  1976),  which  shows  the  number  and  criticality  of  information  transfers  within  each 
iink.  This  tool  is  simpiy  a  listing  of  operator  (or  station)  positions  adjacent  to  half  of  an  N 
by  N  matrix  (example,  Figure  22),  where  N  is  the  number  of  operators  or  stations.  Within 
each  ceil  of  the  matrix  are  the  number  and  criticality  of  specific  link  types  (e.g., 
observations,  ambulatory,  auditory,  etc.). 
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Control  and  Display  panel/console  concepts  are  typically  generated  from  a  variety 
of  system  data;  operational  sequences,  system /mission  requirements,  task  requirements, 
control  and  display  type  selection,  etc.,  and  HFE  design  principles  such  as: 

•  Centrality  -  critical  panel  components  are  placed  centrally  in  an 
operator's  visual  and  operational  sphere 

•  Sequence  of  Operations  -  components  are  placed  according  to  se¬ 
quence  of  functional  use  and/or  reference,  minimizing  visual  and 
motor  transition  times  and  distances 

•  Criticality/Imoortance  -  critical/important  components  are  centrally 
located;  less  critical/important  components  are  located  more  in  the 
operational  periphery 

•  Functional  Grouping  -  functionally  related  controls  and  displays  are 
functionally  grouped 

•  Left  to  Right/Top  to  Bottom  Usage  -  panel  components  should  be 
used  sequentially  from  left  to  right,  or  top  to  bottom 

•  Compatibility  -  controls,  displays  and  whole  panels  should  be  com¬ 
patible  in  terms  of  indexing,  layout,  coding,  etc.,  in  order  to 
minimize  decoding  requirements 

The  object  in  developing  console  concepts  is  to  maximize  panel  design  in  terms  of  these 
principles,  e.g.,  a  panel  offering  sequential  usage  of  components,  centrally  located  critical 
controls  and  displays,  etc.  Some  source  books  on  human  engineering  design  principals  and 
criteria  are  as  follows: 

•  Human  Far  Engineering  Design  for  Army  Material,  MIL-HDBK- 

7 59,  19,. 

•  Naval  Ship  Systems  Command  Display  Illumination  Design  Guide, 

Section  II;  Human  Factors,  NELC,  1973 

•  Guide  to  Human  Engineering  Design  for  Visual  Displays,  Bunker- 

Ramo  Corporation,  1969 

•  Data  Book  for  Human  Factors  Engineers,  Vol.  1  Human  Engineering 

Data,  Man  Factors  Inc.,  1969 

•  Data  Book  for  Human  Factors  Engineers,  Vol.  II,  Common  Formulas. 

Metrics,  Definitions,  Man  Factors,  1969 

•  Human  Engineering  Design  Criteria  for  Military  Systems.  Equipment 

and  Facilities,  MIL-STD-1472,  197 n 

•  Air  Force  Systems  Command  Design  Handbook  1-3  Human  Factors 

Engineering,  AFSC,  1977 

•  Human  Performance  Tradeoff  Curves  for  Use  in  the  Design  of  Navy 

Systems,  APS,  197S 

•  Human  Engineering  Guide  to  Equipment  Design,  Joint  Army-Navy-Air 

Force  Steering  Committee,  1972 

•  Bioastronautics  Data  Book,  NASA,  1973 
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Some  tools  to  assist  in  the  generation  of  workspace  and  panel  concepts  are: 

•  CRAFT  (Computerized  Relative  Allocation  of  Facilities) 

•  WQLAP  (Workspace  Optimization  and  Layout  Planning) 

•  Linear  Programming 

•  CAFES  CAD  (CAFES  Computer  Aided  Design) 

CRAFT 

CRAFT  (Coburn  and  Lowe,  1976)  is  a  computer  program  which  may  be  implemented 
to  identify  optimum  control  and  display  layouts  on  a  panel,  based  on: 

•  Movement  requirements 

•  Frequency  of  control  and  display  use 

•  Control  and  display  distance  (grouping  of  associated  controls  and 
displays).  (The  technique  does  not,  however,  incorporate  control/dis¬ 
play  criticality  in  the  program.) 

•  An  initial  panel  layout  (and  associated  data  such  as  initial  distances, 
control/display  relationships) 

•  Frequency  of  use  for  each  control  and  display 

•  Eye  and  hand  motion  rate  data 

•  Eye  and  hand  workload  data 

The  program  makes  layout  changes  and  computes  cost  factors  (essentially  trade-offs,  e.g., 
extent  of  hand  movement  requirements  vs.  visual  workload).  An  output  of  cost  (or  a 
figure  of  merit)  figures  results  from  the  computation  of  cost  for  all  combinations  of  panel 
layout  (all  possible  control  display  exchanges).  The  technique  can  be  applied  at  the  levei 
of  controls  and  displays,  groups  of  controls  and  display,  subpanels  and  panels.  Therefore, 
after  having  computed  minimum  cost  for  groups  of  controls  and  displays  (i.e.,  for  four 
different  groups),  these  four  may  then  be  input  to  CRAFT  to  determine  minimum  cost  of 
subpanels,  etc. 

WQLAP 

A  program  similar  to  CRAFT  is  WOLAP  (Rabideau  and  Luk,  1974).  The  technique, 
according  to  the  authors,  has  two  advantages  over  CRAFT:  (1)  the  method  "yields  many 
quantitatively  optimized  solutions",  and  (2)  "functional  links  and  sequential  links  are  given 
proper  consideration  in  the  design". 

The  method  of  WOLAP  operation  is  as  follows: 

1.  An  initial  panei  layout  is  evaluated  by  the  program  and  a  cost  figure 
computed. 

2.  Panel  components  are  randomly  rearranged. 
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3.  Cost  is  computed  for  the  arrangement. 

4.  Steps  2  and  3  are  performed  a  prescribed  number  of  times  (by  the  user). 

5.  The  program  retrieves  the  three  layouts  with  the  lowest  cost  and  the 
initial  layout. 

6.  Layouts  and  cost  are  printed  for  four  configurations. 

Figure  23  shows  the  general  flow  of  the  WOLAP  Program. 

In  WOLAP,  cost  is  figured  as  a  function  of: 

•  Transition  distances  (visual,  manual) 

•  Weighting  of  components  that  are  accessed 

•  Probability  of  transitions 

for  all  possible  transitions.  Required  inputs  are  as  follows: 

•  Relative  positions  of  panel  components  in  an  X-Y  plane  (initial 
layout) 

•  Frequency  array  data  table  (data  on  operational  links  of  panel 
components  and  hands,  eyes) 

•  Visual  null  (on  the  X-Y  plane) 

•  Manual  null  (on  the  X-Y  plane) 

•  Tot  ad  number  of  instrument  components 

•  Number  of  iterations  required  (number  of  randomly  generated  con¬ 
figurations) 

•  Relative  weighting  of  controls 

Like  CRAFT,  WOLAP  can  be  implemented  at  the  component,  subpanel  or  panel 

level. 

Tne  above  techniques  are  based  in  part  on  the  manual  Linear  Programming 
technique  reported  by  Freud  and  Sadosky  (1967).  The  manual  approach  uses  control/dis¬ 
play  spacing,  visual  transitions  and  frequency  of  use  as  input  data.  However,  the 
algorithm  reported  was  designed  to  determine  panel  configurations  minimizing  eye  travel 
alone,  the  subsequent  techniques  would  seem  to  have  definite  advantages  due  to  this 
limitation. 

CAD 

The  Computer-Aided  Design  (CAD)  model  of  CAFES  has  been  designed  to  assist  in 
developing  crew  station  configurations  (specifically  cockpit  configurations)  which  are 
consistent  with  mission  requirements,  military  standards  and  specifications,  and  cost  and 
technical  constraints  and  considerations. 
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CAD  is  defined  as  having  three  classes  of  functions:  cockpit  geometry  development, 
cockpit  design  analysis  and  pictorial.  Concept  geometry  development  is  essentially  a 
means  to  deiine  a  workspace  within  the  program,  to  tailor  and  scale  the  workspace 
dimensions,  and  to  allocate  workspace  functional  areas.  Cockpit  design  can  be  analyzed 
to  assist  in  determining  the  suitability  of  any  design  consideration.  This  group  of  CAD 
functions  aids  in  assessing  external  and  internal  vision  characteristics  of  a  given  cockpit 
design  with  a  given  eye  reference  point,  analyzing  and  identifying  potential  reach 
problems  in  a  workspace,  and  determining  if  emergency  escape  can  be  made  from  a 
workspace  by  various  sized  crewmembers,  different  seat  positions,  etc. 

To  use  CAD,  a  variety  of  input  data  is  required,  e.g.,  a  defined  workspace,  which 
can  include  instrument  groups,  control  panels,  controls  (including  reference  point,  shape, 
etc.),  physical  boundaries  and  so  on.  Also  input  are  data  concerning  reach  envelopes, 
scale  factors  (to  modify  sizes  of  workspaces),  eye  reference  points,  transparent  surfaces, 
and  opaque  surfaces. 

CAD  can  analyze  these  geometric  data  and  output  upon  user  request: 

•  A  listing  of  items  which  penetrate  the  escape  envelope  of  predefined 
sizes,  also  providing  the  penetrating  component,  its  level  of  pene¬ 
tration  and  the  item  being  penetrated 

•  Crew  station  Geometry  Data  > 

•  External  vision 

•  Derivations  in  reach  distances  between  reach  limits  and  cockpit 
locations  for  both  hands  and  feet 

•  Vision  distances  from  design  eye  reference  point  to  points  on  a  panel 
surface 

•  Vision  plane  intersection 
Display  Evaluation  Index 

A  method  for  evaluating  control  and  display  effectiveness  is  the  Display  Evaluative 
Index  (DEI)  (Miehle  and  Siegel,  1965).  The  purpose  of  this  technique  is  to  compute  a 
figure  of  merit  regarding  display  utility;  i.e.,  to  provide  an  operator  with  information 
which  can  be  processed  such  that  subsequent  control  activations  will  aid  in  the 
performance  of  a  task. 

The  technique  is  implemented  by  first  identifying  control,  display  and  task  charac¬ 
teristics.  Factors  are  measured  and  a  formula  combines  the  data  to  yield  a  figure  of 
merit.  Inputs  to  the  formula  are  as  follows: 

•  Link  weight  (1  or  2,  depending  upon  amount  of  information  in  an 
informative  transfer  (link),  link  weight  is  a  figure  of  stress) 
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•  Number  of  indicators 

•  Number  of  controls 

•  Number  of  used  controls  and  displays  in  a  task 

•  Total  number  of  instructional  and  information  links 

•  Total  number  of  controls  and  displays 

•  Information  in  digits  (associated  with  control,  display  or  information 
transfer) 

•  Time  for  all  task  transfers 

•  Total  time  for  a  task  to  be  completed 

•  Information  mismatch  (between  control  and  associated  display) 

•  Total  number  of  critical  links 

"The  formula  is  based  on  ten  constructs  and  is  designed  so  that  for  an  'ideal'  system 
the  resultant  index  value  will  be  unity."  To  apply  the  technique  requires  the  construction 
of  a  transfer  chart,  which  is  similar  to  the  S-OSD  and  correlation  matrix.  The  number 
and  presence  of  control  and  display  links  may  then  be  identified.  The  transfer  chart  itself 
may  be  constructed  from  OSDs,  task  analyses,  equipment  concepts  and  control,  display 
and  communications  requirements  analyses. 

While  the  DEI  does  not  predict  human  error  (in  terms  of  a  probability),  it  does 
provide  an  index  of  operability  for  a  panel  or  console  configuration  and,  therefore, 
provides  valuable  trade-off  information. 

APS 

The  Analytic  Profile  System  (APS)  (Siegel,  Fischl  and  Macpnerson,  1975)  is  another 
technique  to  evaluate  the  adequacy  of  displays.  The  technique  requires  that  the  subjects 
make  judgments  concerning  displays  along  the  following  dimensions  (derived  by  multi¬ 
dimensional  scaling): 

•  Stimulus  numerosity 

•  Primary  coding 

•  Contextual/discrimination 

•  Structure  scanning 

•  Critical  relationships 

•  Cue  integration 

•  Cognitive  Processing  Activity 

Judgments  are  made  by  reading  statements  relating  to  each  factor  ("At  first  glance  seems 
to  be  relatively  uncluttered",  relating  to  stimulus  numerosity,  for  example)  and  examining 
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a  display.  The  forced  responses  for  each  display  can  then  be  tabulated  and  the  data  used 
to  identify  areas  of  inadequate  HFE  design  of  displays  (along  the  appropriate  dimensions), 
remedy  the  inadequacies,  or  made  use  the  data  in  trade-off  analysis. 

HECAD 

Aume  and  Topmiller  (1972)  report  a  computerized  design  and  evaluation  tool  called 
HECAD  (Human  Engineering  Computer  Aided  Design).  The  purpose  of  HECAD  is  to 
avoided  the  necessity  of  building  workspace  mock-ups  to  evaluate  complex  design 
concepts. 

HECAD  is  composed  of  two  subprograms,  INDICODE,  which  is  based  on  an  "Index  of 
Electronic  Equipment  Operability"  (developed  by  AIR),  and  DEWO  (Deployment  in 
Workspace).  INDICODE  (like  the  AIR  Data  Store)  is  a  method  of  measuring  workstation 
operability  by  estimating  activation  times  and  reliabilities  of  panel  components  (toggles, 
pushbuttons,  etc.).  The  user  of  INDICODE  specifies,  via  a  CRT  and  lightpen,  the 
components  of  a  panel.  The  progr.^  then  computes  and  prints  the  estimated  time  and 
reliability  of  each.  The  punched  cards  are  used  as  inputs  to  the  second  program,  DEW'O. 
DEWO  produces  a  deterministic  output,  and  also  requires  that  the  user  supply  a  definition 
of  a  single  (3-D)  workspace  to  be  evaluated.  Again  using  the  CRT  and  lightpen,  the 
HECAD  user  arranges  the  individual  components  (50  maximum)  within  the  workspace 
containing  less  than  1 1  panels.  The  data  cards  are  used  to  specify  for  each  component: 

•  The  component  number 

•  Activation  time 

•  Component  dimensions 

•  For  rotating  controls,  the  initial  angular  setting 

Task  sequences  are  then  entered.  The  tasks  are  simply  the  sequences  of  control  or 
display  use  and  are  for  the  sole  purpose  of  determining  visual  or  motor  transition  times,  of 
which  there  are  three  types,  reaching  movements,  turning  movements  (for  rotary  controls) 
and  eye  travel.  Times  are  computed  from  Methods-Time  Measurement  (MTM)  formulas. 
The  end  point  (of  the  eye,  for  example)  for  a  given  task  serves  as  the  starting  point  for  a 
subsequent  visual  task. 

The  simulation  is  simply  an  execution  of  the  tasks  interacting  with  the  components. 
The  program  determines  the  execution  time  of  each  task  (visual,  motor  transitions),  ana 
performance  reliability  (product  of  reliabilities  for  use  of  each  component),  and  counts 
the  number  of  times  each  component  is  used  during  a  task  sequence. 

The  following  are  outputted  by  DEWO: 
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•  Listing  of  panel  equations 

•  Task  sequence 

•  For  each  component 

-  the  identification  number 

-  the  current  location 

-  the  number  of  times  component  is  used  during  a  task  sequence 

-  activation  time 

-  performance  reliability 

•  A  summary  table  containing  a  listing  of  all  actions  during  a  task 
sequence  (by  limb,  start  and  destination  component) 

•  Task  sequence  results 

-  number  of  actions  per  sequence 

-  time  that  hands,  eyes  are  active 

-  communications  times 

-  total  task  time 

-  task  reliability 

•  The  30  longest  transfer  times  are  displayed  on  the  CRT  in  order 

This  last  output  item,  according  to  the  authors,  forms  the  basis  for  redesign 
decisions.  If  the  user  of  HECAD  wishes  to  rearrange  components,  he  can  do  so  simply  by 
using  the  CRT,  light  pen  and  keyboard.  He  may  then  run  the  task  sequence  (or  another 
task  sequence)  again  to  determine  the  effect  of  the  rearrangement. 

TX-105 


A  computerized  technique  similar  to  HECAD  and  developed  by  Boeing  is  TX-105 
(Geer  1976),  which  has  been  developed  to  help  evaluate  workload  of  aircraft  crews  and  to 
evaluate  cockpit  size. 

Three  subroutines  comprise  TX-105,  which  are  used  to  calculate  angles  between  the 
eye  and  points  within  a  cockpit  and  then  to  compute  linear  and  angular  distances  of  eye 
and  hand  movements  during  task  performance. 

Inputs  to  TX-105  include: 

•  Cockpit  geometry  information 

-  control  locations 

-  display  locations 

-  control  and  display  labels 

-  eye  and  shoulder  reference  points 

•  Task  data 

-  name 

-  sequence  of  tasks 

point  to  point  sequence  of  tasks  within  the  workspace 

Outputs  of  the  program  are  similar  to  those  of  HECAD  and  may  be  used  in  the  same 
manner,  as  a  design  tool,  or  to  assist  in  selecting  a  design  concept  which  minimizes  time 
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and  motion  requirements  of  operation.  HECAD,  however,  provides  an  indication  of 
system  operationai  effectiveness  in  terms  of  human  reiiability  (in  addition  to  a  measure  of 
workload) . 

THERP 

A  technique  that  has  been  developed  at  5andia  Labs  and  reported  by  Meister  (1971) 
and  Geer  (1976),  is  known  as  THERP  (Technique  for  Human  Error  Rate  Prediction). 

THERP  is  a  technique  that  predicts  total  system  decrement  as  a  function  of 
estimated  human  error  rates.  The  technique  emphasizes  two  primary  measures,  th* 
probability  of  error  occurrence  and  the  probability  that  error  occurrence  will  result  in 
system  or  subsystem  failure. 

There  are  five  steps  entailed  in  the  use  of  THERP: 

1.  Defining  the  operation 

2.  Establishing  and  listing  tasks 

3.  Estimating  error  rates  per  task 

4.  Predicting  effect  of  errors  on  system  performance 

5.  Deriving  design  modifications  intended  to  minimize  system  failure  rate 

It  is  evident  by  the  final  step  that  THERP  is  intended  to  be  used  as  a  design  tool. 

Sources  of  data  for  THERP  application  are  operational  data  (from  similar  systems 
or,  if  a  redesign  effort  is  underway,  from  the  same  system),  laboratory  studies,  and 
subjective  judgment  (based  on  task  analysis). 

Once  error  rate  and  error  effect  data  are  estimated  (by  whatever  means),  the 
prcoabiiity  of  system  failure  is  estimated  by  compounding  the  data. 

Two  major  assumptions  of  the  technique  are:  (1)  human  errors  that  occur  and  have 
little  or  no  effect  on  system  failure  rate  are  noncritical  (and  weighted  O);  and  (2)  errors 
are  independent. 

For  subjective  data  that  are  used  in  the  technique,  a  set  of  factors  to  be  considered 
in  estimating  error  rates  per  task  are  provided.  These  have  been  termed  Performance 
Shaping  Factors  and  are: 

•  Operator  motivation 

•  Operator  training  experience 

•  Stress  level 

•  Task  difficulty 

•  Task  redundancy 
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•  Manner  and  use  of  job  performance  aids 


Since  the  incidence  of  human  errors  on  system  performance  is  not  considered  to  be  binary, 
the  probability  that  an  error  will  cause  system  failure  must  aiso  be  estimated.  This  could 
be  by  expert  judgment  (design  engineer  perhaps)  or  by  examination  of  failure  mode  effects 
and  criticality  analysis  data. 

TLA-1 

Miller  (1976)  has  described  a  computer  program,  created  at  Boeing,  known  as  TLA- 1 
(Timeline  Analysis  Program  -  Model  1)  which  has  as  its  purpose  to  estimate  operator 
workload  for  task  sequences  within  given  flight  scenarios. 

The  TLA-1  program  is  implemented  in  four  successive  steps.  The  first  is  that  of 
scenario  development.  Here  mission  milestones  are  identified  and  event  times  estimated 
from  mission  flight  plans,  operations,  manuals,  etc.  The  second  step  is  to  derive  task 
data.  Tasks  are  categorized  by  subsystem  and  for  each  task,  identification  is  made  of: 

•  Estimated  task  duration  time 

•  Channel  activity  (left  foot  operated,  right  foot,  hands,  external 
visual,  internal  visual,  cognition,  auditory  or  verbal) 

These  data  are  to  be  derived  or  estimated  from  operator's  manuals,  human  performance 
data  bases  (reach  times,  eye  fixation/rotation  times,  and  so  on),  task  analysis  and  task 
simulation.  The  third  step  in  applying  TLA-1  is  the  development  of  the  task  timeline. 
Worksneets  are  provided  and  tasks  are  sequenced.  For  each  task,  task  name,  identifica¬ 
tion  number,  start  time  and  duration  time  are  coded  on  the  worksheet.  Step  four  is  simply 
to  codify  the  data  in  a  form  suitable  for  keypunching. 

When  the  program  is  run,  the  following  data  are  derived: 

•  Task  time  intervals 

•  Channel  group  workload 

•  Weighted  average  channel  workload  (average  channel  workload) 

•  Mean  workload 

•  Workload  variance 

•  Workload  standard  deviation 

Output  data  (as  requested  by  the  user),  can  be  directed  to  tape  (for  data  storage), 
printer  and/or  a  graphics  plotter.  Printer  output  consists  of: 

•  Mission  Scenario  Report  -  lists  specific  phases  and  associated  events, 
procedures  and  tasks.  This  report  is  quoted  as  having  two  purposes: 

(1)  task  and  procedures  documentation;  and  (2)  verification  that  all 
tasks  are  being  performed  during  a  given  time  interval. 


•  Crewman  Workload  Profile  Report  -  provides  channel,  channel  groups 
and  average  channel  workload  for  each  task  time  interval  (stated  as  a 
percentage.  If  this  figure  is  greater  than  100  percent  either  an  input 
error  has  occurred  or  a  work  overload  condition  exists). 

•  Crewman  Workload  Summary  Status  Report  -  for  each  channel,  chan¬ 
nel  group  and  weighted  average  channel  over  a  phase,  the  following 
are  printed: 

-  mean  workload 

-  workload  variance 

-  standard  deviation 

•  Task-Channel  Activity  Report  -  lists  all  tasks  that  contribute  to  a 
channel  workload  exceeding  a  specified  threshold  (specified  upon 
input  of  data,  70%  for  example). 

•  Subsystem  Activity  Report  -  lists  tasks  that  contribute  to  channel 
overload  ordered  by  subsystem  operation. 

•  Subsystem  Activity  Summary  Report  -  Summarizes  results  of  the 
Subsystem  Activity  Report. 

•  Task  List  Report  -  provides  an  "easy-to-read"  task  catalog. 

Output  of  the  graphical  plotter  includes: 

•  Channel  Activity  Summary  Plot  -  provides  a  bargraph  (for  a  specific 
phase  within  a  mission)  of  channel  workload  mean  and/or  standard 
deviation. 

•  Workload  Histogram  Report  -  plots  (in  a  histogram  form)  channel 
workload,  channel  group  workload  and/or  weighted  average  workload 
as  a  function  of  elapsed  mission  time. 

•  Workload  Summary  Plot  -  bargraph  of  specified  crewmembers  chan¬ 
nel  activity  mean,  standard  deviation  or  weighted  average  channel 
workload. 

•  Mission  Timeline  Plot  -  task  timeline  showing  when  a  task  sequence 
is  in  effect  over  total  mission  time. 


TEPPS 

TEPPS  (Technique  for  Establishing  Personnel  Performance  Standards)  reported  by 
Geer  (1976)  and  Meister  (1971)  is  a  computerized  technique  which  estimates  the 
probability  of  task  accomplishment  and  task  performance  time  (as  THERP  and  HECAD). 
The  technique  is  applied  in  five  steps  using  two  submodels.  The  first  submodel,  the 
Graphic  State  Sequence  Model  (GSSM),  Is  heavily  relied  upon  by  TEPPS.  It  is,  in  essence, 
a  functional  flow  diagram  of  the  ways  in  which  system  requirements  (or  operations)  may 
be  accomplished.  The  second  submodel  is  the  Mathematical  State  Sequence  Mode  (MSSM) 
which  is  a  computer  program  which  handles  the  analysis  of  the  data.  The  MSSM  is  viewed 
as  a  reliability  equation,  that  is,  the  MSSM  is  essentially  a  reliability  block  diagram. 

The  six  steps  of  TEPPS  application  are  as  follows: 
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1.  Describe  the  system 

2.  Develop  the  GSSM  in  terms  of  personnel-equipment  functional  (PEF) 

units 

3.  Determine  predictive  data  for  GSSM  units 

4.  Apply  predictive  data  to  GSSM 

5.  Develop  MSSM  from  GSSM  and  predictive  data 

6.  Implement  computer  program  to  analyze/derive  system  reliability 

Data  for  item  three  above  is  derived  by  a  paired  comparison  technique  which 
estimates  performance  probabilities  and  time  requirements  for  each  PEF. 

The  MSSM  model  (in  effect  the  mathematical  equivalent  to  the  GSSM  with 
associated  success  probabilities)  simply  determines  the  products  of  all  the  PEF  probabili¬ 
ties  of  success  and  sums  PEF  performance  times. 

It  is  possible  to  use  TEPPS  both  as  a  design  tool  and  an  evaluation  tool.  It's  ultimate 
utility  is  probably  that  of  design. 

WAM 

The  Workload  Assessment  Model  (WAM)  of  CAFES  uses  a  timeline  of  mission  tasks 
in  order  to  identify  areas  of  operator  overload.  The  objective  of  WAM  is  to  estimate  the 
effects  on  operator  workload,  due  to  crew  function  allocations,  early  in  a  systems 
developmental  history.  Further,  where  workload  problems  are  revealed  by  WAM,  they  can 
be  lessened  by  functional  reallocations,  increased  automation,  procedural  changes,  etc. 

Procedure  for  WAM  application  is  as  follows: 

•  Prepare  a  mission  profile  and  scenario 

•  Construct  a  mission  phase  chart  (mission  divided  into  phases) 

•  For  each  mission  phase,  identify  tasks  to  be  performed  and  estimate 
task  times 

•  Prepare  a  mission  phase  timeline 

•  Identify  channels  used  for  each  task  (visual,  manual,  cognitive, 
auditory,  verbal) 

•  Prepare  data  for  WAM  execution 

WAM  outputs  tabular  and  plotted  statistical  summaries  of  crewmember  workload  in 
terms  of  channel  activity  per  unit  time  (specified  by  the  user,  six  seconds  is  the  nominal 
recommended  time  segment).  WAM  also  outputs  averages,  standard  deviations  and 
variances  for  channel  workloads  over  all  time  segments  for  each  mission  phase. 

SWAM  (Statistical  Workload  Assessment  Model)  is  a  development  of  WAM  that 
computes  workload  as  a  function  of  required  task  time  vs.  time  available  for  each 
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operator  channel.  SWAM  automatically  identifies  high  workload  conditions  by  computing, 
per  time  segment,  percentage  of  channel  utilization  (percent  of  time  that  a  channel  is 
active  over  the  specified  time  segment  duration),  and  comparing  the  value  (percentage)  to 
an  input  workload  threshold. 

The  SWAM  user  can  optionally  select  a  task  shifting  feature  of  the  program,  e.g., 
where  the  workload  threshold  is  exceeded,  SWAM  will  determine  if  any  tasks  can  be 
shifted  (as  specified  on  input),  without  causing  overloading  in  the  time  interval(s)  to  which 
tasks  are  shif  ted. 

Specific  output  of  WAM  is  as  follows: 

•  Average  channel  workload  for  each  and  combined  channels 

•  Sequenced  list  of  task  start  time,  duration  time  and  end  time 

•  Shifted  tasks  and  amount  of  time  a  task  was  shifted 

•  System  activity  times  (system  activity  defined  by  subsystem  ar 
time,  interval  time,  and  percentage  of  activity  time  for  total  mi: 
time) 

•  List  of  tasks  contributing  to  overload  when  threshold  is  surpassed 

•  Workload  for  each  channel 

•  Workload  for  combined  channels 

•  Workload  standard  deviation  for  each  and  combined  channels  over 
total  mission  time. 

Simulation  Models 

A  group  of  computerized  simulation  models  have  been  developed  by  Siegel  and  Wolf 
(Seigel  and  Federman,  1971;  Siegel  and  Wolf,  1969;  Meister,  1971;  Geer,  1976;  Siegel, 
1977).  These  three  models  are: 

1.  Siegei  -Wolf  1-3  man  model  (SW  1-3) 

2.  Siegel  -Wolf  4-20  man  modei  (SW  4-20) 

3.  Siegel  -Wolf  20-99  man  model  (SW  20-99) 

The  models  are  all  similar  in  terms  of  intended  uses,  inputs  and  outputs.  The  models 
sequentially  simulate  task  performance  of  all  operators.  The  intended  use  (goal)  of  the 
models  is  to  identify  areas  of  operational  overload  (the  models  assume  that  operator 
overload  is  a  basic  element  in  degrading  overall  system  performance).  Stress  is  viewed  as 
a  basic  component  of  overload.  Basic  inputs  to  the  models  are: 

•  Mission  parameters 

•  Time  available  to  complete  tasks 

•  Operator  characteristics  (speed,  stress  thresholds,  motivation,  etc.) 
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•  Task  characteristics 

sequence 

-  essentiality- 
precedences 

-  execution  time 

•  Time  distributions/ task  success  probability  distributions 

The  last  entry  above  makes  the  SW  models  unique  in  character.  In  the  course  of  a 
simulation,  the  time  that  is  required  to  complete  a  task  is  drawn  pseudo-randomly  from  a 
distribution  {normal,  Poisson,  Weiball,  at  the  user's  option;  standard  deviations  are  also 
input  at  the  users  specification  and  option).  Output  consists  of  mission  time  and  success 
distributions  as  a  result  of  updated  mission  simulations  with  outputs  dependent  upon 
pseudo-randomly  drawn  inputs. 

Simply  stated,  the  method  of  simulation  is  thus: 

•  Operator  encounters  a  task  to  perform 

•  Task  urgency  computed  (time  remaining  to  complete  task  sequence) 

•  Stress  computed  (as  a  function  of  urgency) 

•  Task  execution  time  drawn  from  a  distribution 

•  Probability  of  successful  task  completion  drawn  randomly  from  a 
distribution 

•  Data  tabulated  and  stored 

•  Repeated  until  ail  tasks  are  performed 

•  Repeated  until  all  iterations  are  performed 

•  Results  reported 

Stress  enters  into  the  simulation  via  the  determination  of  task  execution  time,  time  left 
to  complete  a  mission  and  task  probability  of  success.  Stress  increases  with  decreasing 
availability  of  time.  Probability  of  successful  task  completion  increases  with  stress  to  a 
point.  As  a  threshold  is  reached  (specified  in  input),  however,  the  probability  of 
successful  task  performance  drops  rapidly  and  task  completion  time  increases.  Prior  to 
stress  having  reached  the  threshold,  the  computer  simulation  may  omit  or  delay  non- 
essential  tasks,  thereby  reducing  stress. 

Since  the  sequence  of  tasks  is  run  many  times  with  various  results  (depending  on  the 
values  drawn  from  the  distributions),  output  data  is,  for  the  most  part,  simply  a  matter  of 
counting  events  and  outcomes,  and  mission  times. 

The  program  produces  as  output  data: 

•  Total  time  expended 

•  Peak  stress  encountered  during  the  simulations 
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•  Probability  of  task  success 

•  Average  waiting  time  (for  another  operator  to  complete  a  task) 

•  Number  of  sub  tasks  ignored 

•  Number  of  tasks  not  successfully  completed 

•  Task  sequence  (or  mission  success)  probability  (successful  task  se¬ 
quences/total  task  sequences) 

The  larger  models  have  some  additional  capabilities  such  as  the  ability  to  simulate 
team  cohesiveness,  aircraft  turbulence,  etc.  Table  3  gives  an  indication  of  the  complexity 
of  the  models  by  listing  input  data  to  the  SW  4-20  man  model  (adapted  from  the  Human 
Reliability  Prediction  Systems  User  Manual,  December  1977). 

Data  sources  for  input  data  are  largely  subjective,  based  on  task  analysis;  however, 
since  probabilities  of  successful  task  performance  and  task  execution  times  are  drawn 
pseudo-randomiy  from  distributions,  it  seems  that  error  may  tend  to  be  minimized. 

HOS 

Another  computerized  technique  that  simulates  human  behavior  in  a  system  is  the 
Human  Operator  Simulator  (HOS)  (Strieb,  Glenn  and  Wherry,  1978,  and  Meister,  1971). 
HOS  is  a  design  and  an  evaluation  tool  that  is  designed  for  use  relatively  early  in  a 
systems  development. 

HOS  simulates: 

•  Information  absorption 

•  Information  recall 

•  Mental  computations 

•  Decision  making 

•  Anatomy  movements 

•  Control  manipulations 

•  Relaxation 

Information  absorption  in  HOS  can  be  made  visually  and  tactually.  As  an  operator 
reads  a  device,  time  is  expended;  when  the  time  expended  (in  terms  of  number  of  micro 
absorptions)  reaches  a  threshold,  the  information  is  deemed  to  have  been  absorbed.  For 
continuous  and  discrete  displays  with  more  than  seven  settings,  an  absorption  error  term 
can  be  introduced  (by  the  user)  and  will  be  used  by  HOS  in  computing  the  operator's 
perceived  value  or  setting  of  the  device  read. 

Information  recall  is  essentially  a  function  of: 
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TABLE  3 

SW  4-20  MAN  MODEL  DATA 


Average  crewmember  aspira¬ 
tion 

Average  crew  pace 
Average  duration  of  scheduled 
event 

Average  psychological  stress 
threshold 

Average  repair  time 
Average  standard  deviation  of 
repair 

Average  standard  deviation  of 
emergency 
Effectivity  of  stress 
Number  of  calories  required  by 
crewmen  (per  day) 

Catnap  length 

Duration  time  of  emergencies 
Duration  time  of  repairs 
Emergency  event  data  set 
Repair  event  data  set 
Number  of  duty  shifts 
Expected  energy  consumption 
Expected  energy  consumption 
for  emergency 
Essentiality  (task) 

Emergency  essentiality 
Essentiality  threshold 
Event  type  number 
Event  number  in  family 
Event  hazard  class 
Event  hazard  class  (emergency) 
Printout  option  indicator  array 
Event  code 
Prerequisite  event 
Equipment  list 

Consumable  rate  of  expenditure 
(umts/hours) 

Consumable  rate  of  expenditure 
(units) 

Consumable  rate  of  expenditure 
(units) — emergencies 
Number  of  repair  events 
Physical  capacitation  fraction 
Derating  constant 
Event  end  type 

Initial  levei  of  consumables 
(units/ hours) 

Initial  level  of  consumables 
(units) 

Threshold  consumables  (units/ 
hours) 

Threshold  consumables  (units) 
Mental  load 


Mental  load  for  emergency 
Maximum  sleep 
Crew  composition  array 
Average  number  of  man  days 
per  incidence  of  physical  inca¬ 
pacitation 

Number  of  iter  ions 
Number  of  days 

Number  of  days  between  emer¬ 
gencies 

Maximum  number  of  days 
Duty  shift 

Equipment  used  array 
Number  of  scheduled  events 
Number  of  men  required  by 
type 

Number  of  men  required  by 
type  for  emergency 
Next  event  number  for  each  al¬ 
ternative 

Average  duration  of  physical 
incapacity 

Percent  fuiiy  qualified  in  pri¬ 
mary  specialty 

Percent  moderately  qualified  in 
primary  specialty 
Percent  unqualified  in  primary 
specialty 

Probability  for  each  alternative 
path 

Cross  training  probability  table 
Equipment  reliability 
Intermittent  reliability 
Repair  touchup  code 
Sea  state/turbuience 
Standard  deviation  of  body 
weight 

Number  of  hours  since  last 
eight-hour  sleep  period 
Percent  fully  qualified  in 
secondary  specialty 
Percent  minimally  qualified  in 
secondary  specialty 
Percent  unqualified  in  second¬ 
ary  specialty 

Earliest  starting  time  allowed 
Fatigue  threshold 
Time  limit  by  which  event  must 
be  completed 

Consumable  threshold  set 

identifier  (units/hours) 
Consumable  threshold  set 

identifier  (units) 


Threshold  set  for  consumables 
below  which  event  is  ignored 
(units/ hours) 

Threshold  set  for  consumables 
below  which  event  is  ignored 
(units) 

Threshold  set  for  consumables 
below  which  emergency  is 
ignored  (units/hours) 

Threshold  set  for  consumables 
below  which  emergency  is 
ignored  (units) 

Number  of  hours  worked  after 
which  no  new  work  assignment 
is  made 

Number  of  hours  worked  after 
which  no  new  work  is  author¬ 
ized 

Mean  body  weight 

Physical  capability  constant 
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•  HAB  strength  -  habit  strength,  the  operators  confidence  in  the 
knowledge  of  a  device's  value  (a  function  of  the  number  of  micro 
absorptions  (time)  in  having  read  a  device) 

•  time  since  a  device  was  read. 

HOS  computes  a  recall  value  (termed  a  probability  of  recall)  and  compares  the 
number  to  one  drawn  randomly  from  a  uniform  distribution;  by  comparing  the  values,  HOS 
determines  if  the  information  is  remembered,  forgotten,  or,  if  the  values  are  sufficiently 
close,  a  new  random  number  will  be  drawn  and  the  comparisons  renewed.  This  method 
applies  only  to  information  absorbed.  HOS  assumes  that  control  and  display  locations, 
methods  of  control  activation,  etc.,  are  always  recalled  (since  HOS  simulates  a  trained 
operator's  performance). 

HOS  simulates  mental  calculations  for  such  things  as  distance  that  can  be  covered 
with  remaining  fuel.  HOS  determines  the  information  required;  if  any  is  not  recalled, 
appropriate  devices  may  be  reread  in  order  to  obtain  the  data. 

Simply  stated,  operator  decisions  are  simulated  by  HOS  by  acquiring  the  data 
required  to  make  the  decision  (as  input  by  the  user).  If  the  information  (and/or  events) 
satisfy  conditions  that  are  required  in  making  a  decision,  the  decision  is  made  and  a  set  of 
appropriate  actions  follow.  !n  certain  circumstances,  where  the  conditions  are  not 
suitable  (e.g.,  a  required  subsystem  or  control  is  not  active),  HOS  will  simulate  the 
operators  behavior  in  enabling  the  subsystem. 

Anatomy  movements  are  also  simulated  by  HOS.  Where  an  operator  movement  is 
required,  HOS  determines  the  channel  to  be  used  as  a  function  of  the  position  of  the 
object,  nominal  and  current  channel  position  (hands  and  feet  positions)  and  the  processes 
of  any  concurrent  body  movements  (if  the  right  hand  is  already  engaged  in  a  control 
activation,  for  example).  For  the  body  part  selected,  HOS  computes  a  "time  charge"  for 
that  action  as  a  function  of  distance  to  move  that  body  part  from  point  A  to  point  8.  If, 
for  example,  the  right  hand  is  engaged,  and  a  movement  is  required  that  cannot  be 
performed  by  the  left  hand,  HOS  will  simulate  the  left  hands  taking  over  the  current  right 
hand  activity,  thereby  freeing  the  right  hand. 

Once  a  control  has  been  accessed  by  an  operator  channel,  control  manipulation  time 
and  effect  is  simulated  by  HOS  according  to  a  variety  of  self  contained  formulas  and  data 
input  by  the  user.  When  body  parts  are  not  active,  HOS  moves  them  to  a  comfortable 
position  thus  simulating  relaxation. 

A  good  deal  of  data  is  required  for  the  use  of  HOS,  including:  mission  scenario  data, 
detailed  task  data,  control  and  display  locations,  method  of  control  activation,  display 
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information,  operator  procedures,  hardware  procedures,  beginning  and/or  nominal  system 
and  operator  status  (hand  locations,  subsystem  activities,  etc.),  and  information  absorp¬ 
tion  times. 


HOPROC  (Human  Operator  Procedures)  is  the  computer  language  which  is  used  to 
define  the  input  data  to  the  computer.  HOS  performs  the  simulation,  analyzes  the  data 
and  provides  output  such  as  the  following: 

•  Timeline  analysis  (the  "snapshot"  interval  of  time) 

•  Channel  loading  within  each  snapshot  interval 

•  Channel  activity  statistics  related  to  each  device 

•  Device  usage  time  of  specific  actions  (time  spent  moving,  manipu¬ 
lating,  recalling,  etc.,  for  each  device) 

•  Link  analysis  (transition  times,  link  frequencies) 

Clearly,  HOS  is  concerned  with  the  time  history  of  the  system,  specifically  as  it 
relates  to  momentary  operator  workload  and  device  usage. 

SAINT 


SAINT  (Systems  Analysis  of  Integrated  Networks  of  Tasks),  as  reported  in  the 
literature  (Ducket  and  Wartman  1976,  Wartman  and  Ducket  1976,  Geer  1971,  Hann  ana 
Kuperman  1976),  uses,  as  its  basic  element,  the  task  to  aid  in  the  design  and  evaluation  of 
developing  systems.  Currently,  three  SAINT  programs  exist  (SAINT  I,  II  and  III).  The 
latter  versions  have  been  improved  and  expanded  from  SAINT  I  to  simulate  changing 
system  conditions  such  as  fuel  remaining,  altitude,  etc. 

To  apply  SAINT,  the  user  must  first  generate  a  task  network  (for  up  to  ten 
operators).  A  procedure  for  generating  these  networks  is  provided;  basic  (stepwise)  inputs 
are: 


•  Identify  sequence  of  tasks  and  task  characteristics 

-  identify  precedence  relationships  (task  dependency) 

-  connect  precedent  relationships  by  key  branches  (connecting 
lines)  which  becomes  the  basis  of  a  network 

-  assign  task  numbers 

-  specify  task  inputs;  is  precedent  task(s)  that  must  be  com¬ 
pleted  prior  to  task  "release"  (note:  first  task  of  a  sequence  is 
labeled  "source  task") 

-  provide  task  description  and  code 

-  specify  task  duration  (can  be  drawn  from  a  distribution  of 
times,  if  desired) 

-  identify  task  outputs  which  represent  a  branching  of  decision 

(1)  deterministic  options 

(2)  probabilistic  (randomly  drawn  from  a  distribution) 

(3)  conditional  — take  ail  options  for  which  conditions  are 
satisfied 
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•  Identify  resources  ("a  non-consumable  commodity  that  is  required  for 
the  performance  of  one  or  more  tasks") 

-  define  resource  availability 

-  identify  resource  by  code 

-  identify  resource  requirements  for  task  completion 

(1)  ail  specified  resources  are  required 

(2)  a  subset  of  all  specified  resources  is  required 

•  Identify  information  attributes 

-  information  flow 

-  information  attributes  and  values 

-  information  requirements  for  each  task 

•  Specify  task  statistics 

-  specify  the  desired  statistics  for  output  (task  completion  time, 
etc.) 

•  Specify  task  priority  (numerical  value) 

•  Identify  resource  attributes 

-  operator  characteristics  (weight,  level  of  intelligence) 

-  equipment  (mode  of  operation,  resistance) 

•  Specify  moderator  functions  (specifies  system  status  variable  that 
may  affect  task  performance  times,  for  example,  waiting  for  another 
operator) 

•  Identify  system  attributes 

-  equipment 

-  equipment  response  times 

•  Specify  state  variables  (fuel  supply  status  over  time,  for  example) 

-  plots  (of  status  over  time) 

-  tables  (of  status  over  time) 

Like  the  Siegel-Wolf  and  other  models,  the  task  analysis  and  subjecxive  judgments 
are  required  to  provide  these  types  of  input  data. 

The  method  simulates  system  and  operator  performance  much  in  the  same  manner 
as  the  Siegel-Wolf  models,  i.e.: 

1.  A  task  is  initiated 

2.  Factors  affecting  task  completion  time  are  examined 

3.  Task  completion  time  (selected  from  a  distribution  or  nominally)  is 
modified  by  those  factors  (waiting,  time  constraints,  etc.) 

4.  Subsequent  task(s)  to  be  executed  is/are  selected  (probabilistically,  as  a 
function  of  priority  or  time,  or  deterministically) 

5.  Factors  affecting  task  "release"  are  surve  “id  (task  dependency,  etc.) 

and  the  sequence  continues  until  the  mission  is  over.  The  mission  itself  is  iterated  a 
specified  number  of  times  and  mission  time  distributions  can  be  developed. 
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The  use  of  SAINT  can  specify  various  outputs  of  the  simulation,  histograms,  plots, 
summary  statistics,  etc.  Of  principle  interest  are  mission  success  data,  task  completion 
time  data  (individual  tasks)  and  mission  times. 

Anthropmetry 

Anthropometry  is  defined  as  the  technology  of  measuring  human  physical  traits, 
primarily  size,  mobility  and  strength  (Hertzberg,  Human  Engineering  Guide  To  Equipment 
Design,  1972).  Parametric  data  for  engineering  use  are  usually  presented  in  percentiles. 
Elimination  of  one  percent  at  both  ends  of  the  distribution  results  in  accommodation  of 
98%  of  the  population.  Designers,  according  to  Hertzberg,  should  attempt  to  accommo¬ 
date  at  least  90%  of  the  population  and  should  strive  for  98%. 

In  the  application  of  anthropometric  data  to  design,  the  same  person  will  differ  in 
terms  of  percentile  for  different  dimensions,  e.g.,  a  man  at  the  5th  percentile  in  terms  of 
arm  reach  may  be  at  the  50th  percentile  in  some  other  dimension.  Bittner  and  Moroney 
(1974)  note  that  the  actual  proportion  of  the  user  population  accommodated  by  a  design 
based  on  using  a  range  of  anthropometric  dimensions  is  not  readily  apparent  due  to 
interactions  among  dimensions.  These  authors  cite  previous  research  which  indicated  that 
the  magnitude  of  the  population  excluded  when  using  a  range  of  dimensions  has  been 
reported  to  be  as  high  as  52%  for  the  1964  population  of  Naval  aviators.  It  was  concluded 
that  design  of  workspaces  without  awareness  of  the  interaction  between  anthropometric 
variables  ultimately  leads  to  a  considerable  reduction  in  the  size  of  the  accommodated 
popuiation. 

Research  directed  at  this  problem  at  the  Naval  Pacific  Missile  Test  Center  initially 
focused  on  surveying  and  evaluating  available  methods  for  calculating  the  accommodated 
proportion  of  the  population.  Table  4  (from  Bittner  and  Moroney,  1974)  presents  the 
methods  identified  and  Table  5  (same  source)  contains  results  of  an  assessment  of  the 
alternate  methods. 

The  latest  anthropometric  source  book  is  that  published  by  NASA  (duly  1978). 
Although  designed  in  part  to  provide  NASA  and  its  contractors  with  material  on  the 
weightless  environment,  it  also  offers  ail  available  anthropometric  data  with  size  range 
projections  for  the  1985  population.  Of  particular  interest  is  material  on  the  variability  in 
human  body  size  which  points  out  to  engineers  the  extent  of  human  body  size  variability 
to  be  considered  in  the  modification  and  design  of  man-machine  systems.  Additional 
information  on  arm-leg  reach  and  workspace  layout,  including  data  for  the  adjustment  of 
workspaces,  etc.,  due  to  anthropometric  differences  and  environmental  conditions  is 
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TABLE  4 

CLASSIFICATION  OF  ACCOMMODATED  PERCENTAGE  METHODS 


(Bitner  and  Moroney,  1974) 
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included.  Volume  II  of  the  same  NASA  publication  summarizes  the  results  from 
anthropometric  surveys  of  61  military  and  civilian  populations  of  both  sexes  from  the 
United  States,  Europe  and  Asia.  It  is  primarily  a  handbook  of  tabulated  dimensional 
anthropometric  data  and  is  the  most  comprehensive  source  of  summarized  body-size  data 
available  at  present.  Volume  III  contains  236  annotated  references  related  to  the  field  of 
anthropometry. 

In  the  design  of  crew  stations,  two-dimensional  drawing  board  manikins  are 
important  aids.  Volume  I  describes  the  USAF  two-dimension  manikin  developed  by  the 
Aerospace  Medical  Laboratory  based  on  1980- 1990  anticipated  body  size  distribution  of 
USAF  fliers.  The  manikins  are  accurate  in  at  least  25  body  size  dimensions  important  in 
the  layout  of  crew  stations.  Consideration  of  variability  in  body  proportions  can  be  taken 
into  account  with  the  use  oi  alternate  limbs.  Additional  sources  of  anthropometrical  and 
related  data  are  as  follows: 

•  Head  and  Neck  Mobility  of  Pilots  Measured  at  the  Eye,  Champion, 

1974 

•  "The  Adult  Human  Hand:  Some  Anthropometric  and  Biomechanical 
Considerations11,  Garrett,  1971 

•  The  Female  in  Equipment  Design,  Giumm,  1976 

•  Muscles:  Testing  and  Function,  Kendall,  et  al.,  1971 

•  Anthropometry  and  Kinematics  in  Crew  Station  Design,  Kennedy, 

•  1972 

•  Designing  for  Muscular  Strength  of  Various  Populations,  Kroemer, 

1974 

•  Muscular  Strength  of  Women  and  Men:  A  Comparative  Study. 
Laubach,  L.,  1976 

•  Statistical  Concepts  m  Design,  McConville,  J.,  and  Churchill,  1976 

•  Engineering  Anthropometry  Methods,  Roebuck,  J.A.,  et  al.,  1975 

•  Anthropometry  of  Air  Force  Women,  Clauser,  et  al.,  April  1972 

•  Anthropometry  of  U.5.  Army  Aviators,  Churchill,  et  al.,  1970 

•  Selected  Anthropometric  Dimensions  of  Naval  Aviation  Personnel, 
Moroney,  et  al.,  1971 

•  The  Body  Size  of  Soldiers,  R.  White  and  E.  Churchill,  1971 

•  Horizontal  Static  Forces  Exerted  by  Men  Standing  in  Common 
Working  Positions,  Robinson,  1971 

•  Anthropometry  of  the  Hands  of  the  Male  Air  Force  Flight  Personnel, 
Garrett,  1970 

•  Anthropometry  of  the  Air  Force  Female  Hand,  Garrett,  1970 

•  Anthropometric  Dimensions  of  Air  Force  Pressure-Suited  Personnel 
Work  Workspace  and  Design  Criteria,  Alexander,  Garrett  ana 
Flannery,  1969 
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•  Databook  for  Human  Factors  Engineers:  Volume  I  -  Human  Engi¬ 
neering  Data.  Kubokawa,  et  al.,  i  969 

•  Clearance  and  Performance  Values  for  the  Bare-Handed  and  Pressure 
Gloved  Operator,  Garrett,  1963 

CAR 

Several  computerized  methods  to  determine  if  an  operator  can  fit,  anthropo- 
metrically,  into  a  workspace  are  available.  CAR  (Crewstation  Assessment  of  Reach) 
(Geer  1976,  Bittner  1976)  was  developed  as  a  Monte  Carlo  model  for  examining  pilot 
anthropometric  data.  The  model  entails  a  link  man  model  and  an  adjustable  workspace 
model.  Given  the  workspace  model,  CAR  computes  the  percentage  of  aviators  that  can 
be  accommodated  by  that  workspace  (cockpit).  CAR  provides  two  submodels  to  the  user, 
(1)  Monte  Carlo  Simulation  Model  (MCSM),  and  (2)  Crewstation  Analysis  Model  (CAM). 

The  MCSM  option  generates  sample  aviator  anthropometric  data.  MCSM  randomly 
generates  12  anthropometric  measures  for  a  user  specified  number  of  sample  aviators. 
These  measures  are  translated  into  19  man-model  links. 

CAM  evaluates  a  deferred  crewstation  geometry  using  crewmen  sample  generated 
by  MCSM.  Output  is  the  percentage  of  crewmen  that  can  be  accommodated  by  the  input 
crewstation  geometry. 

The  cockpit  analysis  model  determines  the  percentage  of  population  excluded  based 
on  geometric  parameters  of  the  workspace.  Components  ox  this  model  are: 

1.  Pilot  link  system 

2.  Pilot  sample  generator 

3.  Seat-cockpit  layout 

4.  The  testing  component 

COMBIMAN 

Evans  (1976)  has  reported  a  computerized  technique  known  as  COMBIMAN  (Com¬ 
puterized  Biomechanical  Man-Model)  which  is  a  design  aid  to  anthropometrically  fit 
operators  to  workspaces.  COMBIMAN  was  built  to  aid  in  the  design  and  evaluation  of 
aircraft  workspaces  but  claims  to  be  applicable  to  any  workspace.  Specific  applications 
are: 

•  The  evaluation  of  existing  workspaces 

•  Design  and  evaluation  of  new  workspaces 

•  Personnel  selection  criteria  for  workspaces 

•  Mapping  of  external  visability  plots 
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COMBIMAN  consists  of  two  submodels,  the  man/model  and  the  workspace  design 
model.  The  man/modei  consists  of  a  33  iink  skeletal  system  where  link  length  can  be 
specified  either  by  the  user  or  automatically  via  reference  to  an  anthropometric  data 
base.  The  workspace  model  permits  the  development  of  a  three  dimensional  workspace 
containing  operating  panels  of  three  to  six  vertices.  The  workspace  can  be  established 
using  either  punched  cards  or  use  of  a  CRT,  light  pen  and  keyboard. 

User  supplied  data  (in  addition  to  workspace  dimensions)  can  be: 

•  Direct  anthropometric  measures  of  subjects 

•  Data  base  percentages 

•  Combinations  of  measures  and  data  base  measures 

•  Required  population  dimensions  (to  fit  a  workspace) 

•  Required  or  established  maximum  rotational  angles 

•  Bodily  restrictions  such  as  clothing 

The  three  important  subprograms  of  COMBIMAN  are:  (1)  the  interactive  graphics 
program  (output  program);  (2)  the  COMBIMAN  Anthropometric  Data  Base  Maintenance 
Program  (CBMAN);  and  (3)  the  COMBIMAN  Workspace  Data  Base  Maintenance  Program 
(CBMWM). 

The  basic  outputs  of  COMBIMAN  are  indications  of  successful  or  unsuccessful 
reaches,  given  a  specific  workspace  and  input  anthropometric  data  of  an  operator.  The 
dimensions  of  the  man/model  may  be  varied  using  the  keyboard  and  light  pen,  thereby 
determining  minimum  and  maximum  reach  distances  of  the  simulated  operator. 

CGE 

The  CGE  model  of  CAFES  is  used  to  identify  and  analyze  cockpit  reach  characteris¬ 
tics  and  test  cockpit  compliance  with  military  specifications  and  standards.  CGE  is 
applied  in  two  steps.  First,  input  data  are  prepared,  including: 

•  Cockpit  geometry  data 

•  Controls  data 

•  Eye  reference  points  data 

•  Task  sequences  data 

•  Control  shapes  data 

Outputs  of  the  CAD  modei  of  CAFES  can  be  implemented  as  partial  input  to  CGE.  The 
second  step  entails  specifying  output  for  the  DMS/CGE  interface  model. 

CGE  uses  mathematical  routines  to  simulate  activities  of  a  variable  sized  man- 
model.  Output  of  CGE  includes: 
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•  Physical  and  visual  interferences 

•  Unfeasible  reach  tasks 

•  Crew  station  compliance  with  military  specifications  and  standards 

•  Feasible  tasks  accomplished 

ORACLE 

Operators  Research  and  Critical  Link  Evaluation  (ORACLE)  (Meister,  1971)  is  a 
computerized  diagnostic  and  evaluation  tool  of  workload.  The  model  simulates  the  input 
and  processing  rates  of  information  nodes  and  links  in  an  information  flow  system. 
ORACLE  is  not  behaviorally  oriented,  but  may  have  application  to  man/machine  systems 
if  an  assumption  is  made  that  nodes  may  be  modeled  to  represent  human  operations. 

According  to  the  developers,  the  uses  of  ORACLE  are: 

•  The  determination  of  the  number  and  types  of  personnel  required  for 
a  task  mixture  (man/machine  allocations)  and  system  configuration 

•  The  determination  of  design  change  effects  on  system  effectiveness 

•  The  identification  of  critical  elements  (paths)  in  an  operational 
sequence 

•  Measurement  of  the  effect  of  degradation  of  individual  system 
functions 

Input  data  to  ORACLE  include: 

•  Input  rates  for  information  units  (messages  per  unit  time) 

•  Message  initiation  times 

•  Message  response  times 

•  Message  priorities 

•  Probability  of  an  events  occurrence  based  on  equipment  availability 
and  reliability  criteria 

The  data  are  used  by  the  program  to  provide  a  timeline  history  of  system  operations. 
Specific  outputs  include  a  prediction  of  total  processing  time  required  for  a  given 
sequence  of  events  and  the  identification  of  queues  of  information  representing  informa¬ 
tion  overload  at  nodes. 

HFTEMAN/HEDGE 

Two  guides  for  planning,  implementing  and  analyzing  HFE  Test  and  Evaluation  are 
HFTEMAN  (Human  Factors  Test  and  Evaluation  Manual)  (Malone  and  Shenk,  1977)  and 
HEDGE  (Human  Factors  Engineering  Data  Guide  for  Evaluation)  (Malone  and  Shenk,  197S). 

HFTEMAN  is  primarily  directed  towards  developing  an  HFE  T&E  plan.  HFTEMAN  is 
divided  into  three  volumes: 


•  Data  Guide  -  contains  guidelines  concerning  what  to  evaluate  for 
classes  of  equipment  and  types  of  tests 

•  Support  Data  -  contains  criteria  expanding  the  guidelines  and  criteria 
in  the  Data  Guide 

•  Methods  and  Procedures  -  contains  guidance  on  how  to  design,  set  up, 
conduct,  and  analyze  data  obtained  in  implementing  the  HFE  TdcE 
plan 

The  Data  Guide  provides  eight  steps  to  perform  an  HFE  OTdcE  and  ten  steps  to 
perform  an  HFE  DTdcE.  The  first  seven  steps  are  identical  for  both  and  are: 

L.  Inspect  the  test  item  and  review  documentation 

2.  Identify  the  type  of  test(s)  to  be  performed  —  test  types  are: 

•  Operability 

•  Maintainability 

•  Transportability 

•  Habitability 

•  Portability/usability 

•  Erectability 

3.  Identify  the  class  in  which  the  test  item  belongs  —  equipments  are 
classed  in  the  Data  Guide  as  being: 

•  Vehicles  (land,  sea,  air) 

•  Weapons  (individual,  missiles,  etc.) 

•  Electrical  optics 

•  Support,  supply  and  service 

•  Personnel  support 

4.  Identify  pertinent  use  conditions  to  be  considered  as  test  conditions 

5.  Identify  user  activities  and  tasks 

6.  Identify  equipment  components  associated  with  user  tasks 

7.  Identify  potential  HFE  problem  areas  associated  with  equipment  com¬ 
ponents 

The  final  step  for  HFE  OTdcE  is: 

8.  Prepare  a  checklist  or  questionnaire  to  be  used  in  observing  or  sampling 
fleet  personnel  performing  with  the  item 

The  final  steps  for  HFE  DTdcE  are: 

9.  Select  criteria  (from  support  data  volume) 

10.  Select  test  methods  (methods  and  procedures  volume) 

11.  Formulate  HFE  test  plan  using  selected  tests  and  test  criteria 

Design  criteria  for  HFE  considerations  such  as  location  and  arrangement,  sizes  and 
shapes,  direction  and  force,  information,  visibility,  use  conditions,  and  safety  are  provided 


3-69 


1 


by  equipment  components  (for  example,  controls,  displays,  workspace,  doors,  hatches, 
passageways,  etc.).  These  criteria  are  provided  in  the  Data  Guide  and  Support  Data 
volumes. 

The  methods  and  procedures  volume  contains  data  on  how  to  conduct  various  HFE 
T&E  evaluations.  Specifically,  this  volume  is  comprised  of  the  following  evaluations: 

•  Design  Evaluations 

-  visibility 

-  speech  intelligibility 

-  workspace  and  anthropometries 

-  force  torque  measurements 

•  Performance  Evaluations 

-  task  checklists 

-  error  likelihood  analysis 

-  team  performance  evaluations 
training  evaluation 

•  Maintainability  Evaluations 

-  equipment  and  facilities 

-  maintenance  safety 
maintenance  information 

-  maintenance  actions 

-  accessibility 

•  Habitability  Evaluations 

-  lighting 

-  noise  measurements 

-  toxic  hazards 

-  environmental  measures 

-  vibration  measures 

HEDGE  was  developed  for  the  U.S.  Army  Test  and  Evaluation  Command  (TECOM) 
and  defines  the  Test  Operating  Procedures  (TOPs)  for  the  evaluation  of  Army  materiel. 
HEDGE  is,  however,  applicable  to  a  large  variety  of  equipments. 

HEDGE  is  divided  into  two  parts.  Part  1  defines  a  number  of  HFE  T&E  procedures. 
Part  2  contains  design  criteria  (in  much  the  same  manner  as  HFTEMAN).  HEDGE 
describes  the  preparation  for  an  HF  equipment  evaluation  and  provides  specific  test 
procedures  for: 

•  Lighting  evaluation 

•  Noise  measurement 

•  Vibration  measurement 

•  Atmosphere  composition  measurement 

•  Temperature,  humidity  and  ventilation  measurement 

•  Visibility  measurement 

•  Speech  intelligibility 


3-70 


•  Workspace  and  anthropometries  measurements 

•  Force/torque  measurements 

•  Design  checklists 

•  Panel  commonality  analysis 

•  Maintainability  assessment 

•  Individual  performance  assessment 

•  Error  likelihood  analysis 

•  Crew  performance  assessment 

•  Information  systems  assessment 

•  Training  assessment 

•  Workload  assessment 

•  Task  checklists 

•  Questionnaires  and  interviews 

•  Dexterity 

•  Clothing  and  equipment 

HFTEMAN  and  HEDGE  represent  roughly  22  separate  HFE  T&E  techniques  that 
have  been  grouped  into  an  entire  HFE  T&E  procedure.  These  two  procedures  also  provide 
a  series  of  data  collection  forms  as  an  aid  to  the  implementation  of  individual 
measurement  and  analytic  techniques.  Sample  task  checklists,  design  checklists  and 
questionnares/interviews  are  also  provided. 

HFE  design  checklists  are  used  as  a  tool  to  identify  areas  where  HFE  design  criteria 
and  HFE  design  principals  have  been  violated.  Checklists  are  constructed  from  sources 
such  as  MIL-STD-1472,  HFTEMAN,  HEDGE,  Requirements  Analysis,  and  so  on.  Examples 
of  checklists  are  provided  in  Tables  6  through  11  (from  Malone  and  Shenk,  1978).  HEDGE 
and  HFETEMAN  provide  detailed  criteria  for  system  components  relating  to: 

•  Labels,  manuals  and  markings 

•  Steps,  ladders 

•  Railings,  handholds 

•  Doors,  hatches 

•  Controls 

•  Displays 

•  Workspace 

•  Communicators 

•  Handles 

•  Optics 
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TABLE  6 

HEDGE  TRAINING  DEBRIEFING  QUESTIONNAIRE 


Test  Title  Date 

Name  Grade/Rank 


1.  List  your  military  occupational  specialty  (MOS). 

2.  How  long  have  you  had  this  military  occupational  specialty? 

_  years 

_  months 

3.  In  the  space  below,  list  school  training  you  have  had  in  this  military  occupational 
specialty. 


4.  Which  test  item  component  did  you  use? 

5.  Did  you  encounter  any  problems  during  the  test  which  you  Yes  No 

attribute  to  insufficient  training?  if  yes,  explain. 


6.  Did  you  understand  all  phases  of  your  training?  If  not, 
explain. 


Yes  No 
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TABLE  6 
(Continued) 

Do  you  think  it  takes  any  special  skill  to  operate  the  equip-  Yes  No 

ment  you  used?  If  yes,  state  the  special  skill. 


Would  you  like  any  additional  training  on  the  test  item  Yes  No 

before  you  are  assigned  to  operate  it  in  a  tactical  unit? 

If  yes,  specify  additional  training. 


State  in  your  own  words  how  your  training  could  be  improved. 
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TABLE  7 


PRELIMINARY  HF  ANALYSIS 


Test  Item 

Fvnluntnr 

Sration  _ 

Date 

_ 

Factors  to  be  analyzea  in  the  HFE  Subtest 

Selected  Tasks 
(from  Task  Checklist) 

environmental  1  Equipment  Test  Participant 

Conditions  1  Characteristics  i  Characteristics 

Performance 

Measures 

i 


COMMONALITY  ANALYSIS  DATA  FORM 


tot  uiii<|uc  cuhI  present  on  only  oik;  panel  -  irulicutc  Unit  pone  I  o.">  A  or  IJ. 


TABLE  10 

DESIGN  CHECKLIST 
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TASK  CHECKLIST 
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•  Operating  elements,  etc. 


for  HFE  considerations  such  as:  location  and  arrangement,  sizes  and  shapes,  directions 
and  forces  required  to  operate,  clearances,  visibility,  environment  and  use  conditions, 
safety  and  operational  conditions.  They  also  contain  detailed  guidance  on  the  method  of 
checklist  application,  data  reduction  and  data  analysis. 

Task  checklists  (example  in  Table  11)  are  used  to  evaluate  operator/maintainer  task 
performance  while  the  operator  is  engaged  in  job  performance.  The  checklist  itself  is 
simply  a  sequential  listing  of  tasks  to  be  performed,  with  space  available  on  the  form  to 
check  (1)  adequate  task  performance,  (2)  inadequate  task  performance,  or  (3)  task  not 
performed.  For  tasks  that  have  been  inadequately  performed,  the  evaluator  can  make 
comments  concerning  potential  problem  areas  which  may  contribute  to  that  performance. 
The  checklist  itself  is  relatively  simple  to  construct,  but  application  requires  both  a 
moderate  to  high  level  of  HFE  experience  and  an  understanding  of  the  equipment. 
Further,  tasks  may  proceed  at  a  rate  faster  than  an  evaluator  may  be  able  to  monitor 
performance,  respond  to  the  checklist,  and  make  notes  regarding  task  performance  and 
operator  comments.  Videotape  recordings  (such  as  those  proposed  in  Task  Analysis 
Reduction  Technique  (TART))  can  be  of  great  utility  in  applying  task  checklists. 

Error  Analysis 

Error  Analysis  techniques  are  used  in  performing  trade-offs  or  identifying  areas 
where  redesign  of  equipment  or  procedures  is  required.  The  purposes  of  error  analysis  are 
to  identify  areas  where  design  concepts  may  tend  to  reduce  operator  reliability  in  critical 
functional  areas.  Three  analyses  are  of  particular  interest,  the  Task  Analysis  approach, 
the  OSD  approach  and  Equipment  Error  Probability  Analyses  (Malone,  1976). 

The  Task  Analysis  and  OSD  approaches  address  procedural  errors  and  potential 
consequences.  The  procedure  entails  examining  each  task,  function  or  activity  in  order  to 
identify  potential  errors.  For  each  potential  error  identified,  assessments  are  made 
concerning: 

•  Impact  of  error  occurrence  on  system  or  mission  failure  probabilities 

•  Operator  safety  as  a  result  of  error  occurrence 

•  Degree  to  which  equipment  design  can  have  positive  influence,  error 
likelihood  and/or  mission  reliability 

•  Degree  to  which  procedural  design  can  reduce  error  occurrences  and 
enhance  mission  reliability 

Assessments  of  equipment  error  probabilities  are  made  for  controls  and  displays 
according  to  estimations  of  error  types. 
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Control  errors  are  described  as  being: 

•  Inadvertant  actuation 

•  Substitute  -  incorrect  selection  of  a  control 

•  Activation  -  incorrect  setting 

•  Temporal  -  control  activator  at  incorrect  time 

•  Sequential  -  operating  control  out  of  sequence 

•  Omission  -  failure  to  operate  a  control 

Display  errors  include: 

•  Reading  -  misreading  a  display 

•  Substitution  -  reading  the  wrong  display 

•  Interpretation  -  correctly  reading  but  misjudging  the  displayed  infor¬ 
mation 

•  Omission  -  failure  to  receive  the  displayed  information 

Judgments  are  required  to  estimate  probability  of  each  error  dimension  for  each 
control  or  display  according  to  (1)  high  probability  of  error,  (2)  moderate  probability,  or 
(3)  low  probability  of  error.  Error  criticality  estimates  are  also  made  and  are  estimated 
to  be  high,  moderate  or  low  criticality. 

Controls  and  displays  that  are  of  high  or  moderate  criticality  and  which  have 
moderate  or  high  probability  of  error  occurence  on  any  dimension  are  considered  as 
requiring  both  additional  evaluation  and  redesign.  An  example  of  error  likelihood  analysis 
format  is  shown  in  Figure  24  for  controls  and  Figure  25  for  displays. 

Functional  Description  Inventory 

Heim  (1976)  has  described  the  Functional  Description  Inventory  (FDD  as  a  tool  to 
quantifiabiy  assess  the  effectiveness  of  man/machine  interfaces.  The  procedure  requires 
the  analysis  of  the  operational  functions  of  each  crewmember.  The  functions  are 
hierarchically  determined  by  roles,  duties  and  tasks  performed  by  each  operator. 
Operator  judgments  for  roles,  duties  and  tasks  are  selected  to  determine  average 
crewmember  judgments  towards: 

•  Importance  for  mission  success 

•  Frequency  of  performance 

•  Training  adequacy 

•  System  effectiveness 

For  each  role,  duty  and  task,  crewmembers  are  requested  to  respond  to  each  of  the  four 
items  above  on  a  scale  of  five. 
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Data  are  analyzed  by: 

•  Rank  ordering  responses  (importance,  frequency,  training  effective¬ 
ness  and  system  effectiveness)  for  roles,  duties  and  tasks 

•  Computing  frequency  distribution 

•  Computing  mean  and  standard  deviations  of  responses  for  individual 
roles,  duties  and  tasks 

Further,  standard  scores  can  be  compounded  (criticality  and  frequency  of  use,  for 
example)  reflecting  the  combined  weights  of  the  duties  rated  in  these  parameters. 

Development  of  the  technique  was  continued  by  O'Conner  (1977)  who  termed  it  a 
Decision  Analytic  Technique  (Figure  26).  A  rating  scale  was  developed  (after  role  duty, 
task  hierarchy  development)  emphasizing  workload  and  system  effectiveness.  Pilots  (F-1S 
aircraft)  rated  both  tasks  along  the  workload  dimensions  and  system  effectiveness.  A 
paired  comparison  technique  was  used  to  weight  task  criticality.  Both  FD1  and  Design 
Analytic  Technique  are  part  of  the  Mission  Operability  Assessment  Technique  (MOAT) 
being  developed  at  the  Pacific  Missile  Test  Center. 

F&E  Kits 

HFE  Kits  and  a  variety  of  measurement  tools  are  available  to  aid  in  HFE  T&E.  One 
particular  kit  which  has  been  assembled  by  Perceptronics  includes  the  following: 

•  Sound  level  meter  and  analyzer 

•  Vibration  meter  and  analyzer 

•  Photometer 

•  Spot  brightness  meter 

•  Force/torque  and  dimension  kit 

•  Portable  weather  station  (with  readouts  that  can  be  located  away 
from  the  actual  measuring  devices.  Readouts  indoors,  for  example, 
while  the  weather  station  is  out-of-doors) 

•  Hot  wire  anemometer 

•  Aspirating  Psychrometer 

•  Thermometer  (digital) 

•  Gas  tester  (universal) 

•  Monitoring  gas  sampler 

•  Anthropometry  instrument  kit  (goniometers,  tapes,  etc.) 

•  Digital  time 

•  Event  counter  (multiple) 

•  Camera  (Polaroid  SX-70) 

•  Video  recording  system 
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•  Audio  tape  recorder 

•  Instrumentation  tape  recorder 

•  Scientific  calculator 

•  Digital  test  meter 

•  Tool  kit 

•  Battery  charger 

Additional  tools  that  may  comprise  an  HFE  kit  are  film  editors,  surface  pyrometers 
(for  roughly  100°F  plus  surface  temperatures),  anthropometric  positioners,  movie  cam¬ 
eras,  projectors,  screens,  portable  radio  systems,  tripods,  etc. 

Many  of  the  individual  tools  of  an  HFE  kit  are  used  during  actual  operations  and 
hence  a  goal  is  to  select  tools  that  are  as  unobstrusive  as  possible,  so  as  not  to  effect 
operations  and  potentially  corrupt  the  data. 

TART 

The  Task  Analysis  Reduction  Technique  (TART)  (Ellis,  1970)  serves  as  an  aid  to 
evaluating  workstation  designs  using  either  mockups  or  real  equipment.  The  purpose  of 
TART  is  to  minimize  loss  when  quantifying  performance  and  to  improve  the  usability  of 
the  qualitative  form  of  the  data;  that  is,  to  increase  the  identity  between  actual  task 
performance  and  data  that  describe  it. 

The  tool  'employs  a  video  recording  system  for  data  collection,  a  video  playback 
device  and  Task  Analysis  -  Operational  Sequence  diagrams  for  analysis.  The  actual  steps 
in  TART  application  are  as  follows: 

•  Develop  TA/OSDs 

•  Video  tape  task  performance  (using  mockups  or  actual  equipment) 

•  Establish  a  timeline  using  video  playback 

•  Analyze  a  timeline  to  determine: 

task  frequency 

-  task  loading 

-  sequential  task  impulses 

PAARS 

Personnel  Activity  Analysis  Radio  System  (PAARS)  (Potema,  1969)  is  a  field  testing 
technique  to  collect  job  activity  information  and  data  via  a  radio  system.  Radios  are  used 
in  the  technique  by  one  (or  more)  HF  analyst  during  operator  task  performance.  As 
applied,  operator  activities  at  various  workstations  can  be  timed,  phased,  sequenced  and 
task  checklists  can  be  applied.  PAARS  allows  tape  recordings  to  be  made  during 
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operations.  The  technique  requires  a  base  station  and  a  supply  of  waikie  talkie  type 
radios.  These  can  be  either  used  by  actual  operators  or  HF  analysts.  Application  requires 
that  an  operator  (or  observer)  vocalize  tasks  during  performance.  PAARS  may  be 
valuable  in  generating  or  verifying  OSDs,  establishing  timelines. 

Human  Reliability 

A  technique  for  the  Allocation  of  Man-Machine  Reliability  is  described  in  the 
Human  Reliability  Prediction  System  Users  Manual  (1977).  The  purpose  of  the  technique 
is  to  permit  equipment  and  human  operator/maintainer  (reliability)  to  be  addressed  as  part 
of  design  trades.  The  basic  concept  is  Operational  Reliability,  of  which  the  human 
operator  is  viewed  as  an  integral  part. 

Steps  in  applying  the  technique  are:  (1)  to  identify  critical  functions  and  reliability 
and  maintainability  design  parameters  (in  effect,  construct  a  function  probability  tree), 

(2)  identify  constraints  to  allocations  for: 

•  Minimum  mission  reliabilities 

•  Minimum  operational  readiness 

•  Maximum  cost  (life  cycle,  acquisition,  support) 

•  Personnel  (number,  skill  level  requirements)  and 

(3)  maximize  the  equation: 

Operational  Reliability  =  (P^  .  r^);  where 

P-  =  probability  of  a  mission,  and 
r-  =  mission  reliability. 

The  technique  is  stated  to  have  application  to  performance  of  maintenance  or 
operability  trade-offs,  e.g.,  the  essential  tradeoffs  on:  (1)  automation  and  human 
maintenance  reliability,  versus  (2)  manual  system  and  human  operational  reliability, 
versus  (3)  equipment  reliability  of  various  levels  of  automation.  A  simplified  dynamic 
programming  procedure  for  maximizing  the  equation  is  given  and  provided  in  Figure  27. 

A  technique  for  predicting  the  probability  of  maintenance  task  success  is  compound¬ 
ing  (Human  Reliability  Prediction  Systems  Users  Manual,  1977).  Three  steps  are  involved 
in  applying  the  technique: 

1.  Multidimensional  scaling 

2.  Individual  performance  index  computations 

3.  Reliability  index  computation 


Multidimensional  scaling  involves  the  identification  of  tasks  involved  in  system 
maintenance  activities.  A  factor  analysis  was  performed  and  general  job  factors  emerge. 


START 


SELECT  PARAMETERS 
(FAILURE  RATES,  REPAIR  RATES) 


FIGURE  27 

PROCEDURE  FOR  FUNCTIONAL  ALLOCATION  3ASED  ON 
RELIABILITY  ESTIMATES  AND  RELATIVE  COSTS 


A  factors  analysis  for  electronics  systems  is  performed  and  factors  (said  to  be  applicable 
to  ail  electronic  system  maintenance  activities)  are  reported  as  being  eiectro-cognitive, 
electro-repair,  etc.  Once  job  factors  are  identified,  personnel  performance  index  (PPI) 
computations  are  derived  according  to  tasks  in  the  scaling  (factors).  This  entails 
identifying  usually  effective  (UE)  and  usually  ineffective  (UI)  maintainer  performance  on 
a  series  of  task  performances.  The  performance  index  is  then  calculated  by  the  following 
formula: 


PI 


UE 

UE  ♦  UI 


which  is  simply  the  ratio  of  successful  performances  over  total  rated  observations.  With 
the  availability  of  performance  indices  for  each  maintainer  for  each  factor,  human  reli¬ 
abilities  for  individual  malfunctions  of  a  system  can  be  estimated  by  computation. 
Formulas  for  estimating  human  reliability  are  provided  for: 

•  Series  Maintenance  Activities  -  simply  determine  product  of  per¬ 
formance  indices 

•  Parallel  Maintenance  Activities  -  where  two  or  more  maintainers 
perform  independently 

•  Complex  Maintenance  Activities  -  where  task  dependency  exists  be¬ 
tween  two  or  more  maintenance  technicians 


The  data  to  apply  the  technique  to  electronic  systems  are  in  Table  12  and  these  data 
are  said  to  be  applicable  to  any  such  system.  For  other  systems,  multidimensional  scaling 
will  have  to  be  performed  in  order  to  apply  the  technique  (e.g.,  hydro-repair,  hydro¬ 
safety,  etc.).  Data  to  perform  the  factor  analysis,  as  well  as  the  determination  of  PPIs,  is 
reported  as  being  provided  by  judgments  of  experienced  personnel  and  presumably  the 
same  or  a  similar  method  could  be  undertaken  for  hydraulic  or  mechanical  systems. 


MIL-HDBK-472,  Maintainability  Prediction  (1966)  provides  four  procedures  for 
estimating  system  maintainability.  The  procedures  each  address  one  of  four  separate 
maintainability  issues,  as  follows: 

Procedure  1:  Predicts  system  downtime  of  airborne  electronic  and  elec¬ 

tromechanical  system  involving  moael  replacement  of  com¬ 
ponents. 

Procedure  2:  Predicts  corrective,  preventive  and  active  maintenance 

parameters  (specifically  maintenance  time) 


Procedure  3:  Predicts  maintainability  of  ground  electronic  systems  and 

equipments  (mean  downtime) 

Procedure  4:  Predicts  system  downtime  as  a  function  of  system  history 

and  subjective  judgments 
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TABLE  12 

PERSONNEL  PERFORMANCE  INDEXES 
(HUMAN  RELIABILITY  PREDICTION  SYSTEM  USER'S  MANUAL,  1977) 


Job  Activity 

Electro  Cognition 
Electro  Repoir 
Instruction 
Electro  Safety 
Personnel  Relationships 
Electro  Circuit  Analysis 
Equipment  Operation 
Using  Reference  Material 


Career  Field  (Navy) 


EM 

ET 

FT 

1C 

RD 

RM 

ST 

TM 

.55 

.83 

.86 

.62 

.33 

.63 

.92 

.36 

.78 

.99 

.92 

.70 

.30 

.71 

.70 

.40 

.75 

.95 

.97 

.45 

.57 

.95 

.51 

/  / 

.00 

.50 

.98 

.95 

.65 

.92 

.70 

.42 

.62 

.74 

.70 

.79 

.63 

.40 

.77 

.35 

o 

00 

.53 

.90 

.95 

.58 

.40 

.65 

.74 

.60 

.92 

.85 

.95 

.65 

.90 

.85 

.92 

.75 

.73 

.90 

.87 

.62 

.95 

.92 

.88 

.66 

Electrician's  Mate 

Electronics  Technician 

Fire  Control  Technician 

Interior  Communications  Electrician 


RD  -  Radarman 
RM  -  Radiomen 
ST  -  Sonar  Technician 
TM  -  Torpedoman's  Mate 


s 


Table  13  provides  basic  information  for  each  procedure  concerning  its  application, 
measures,  inputs  and  reliability. 

ERUPT 

Two  techniques  have  been  reported  by  Beek  (1967):  (i)  ERUPT  (Elementary 

Reliability  Unit  Parameter  Technique)  and  (2)  Multiple  Correlation  Approach  to  the 
identification  of  personnel  characteristics  which  show  greatest  effects  of  system  failure 
and  repair  rates. 

ERUPT  is  a  technique  for  the  estimation  of  two  total  system  readiness  parameters: 

1.  Probability  that  a  failure  is  detected  and  repaired 

2.  Probability  that  maintenance  does  not  induce  failures 

In  ERUPT,  systems  components  are  grouped  into  ERUs  (Elementary  Reliability  Units), 
which  become  the  units  by  which  reliability  of  the  total  system  is  estimated.  Each  ERU 
has  a  reliability  which  is  estimated  from:  probability  of  failure;  probability  of  storage 
failure;  probability  of  repair  (while  in  storage  or  in  operation);  and  a  probability  of 
failures  induced  during  maintenance.  Total  system  reliability  is  considered  to  be  the 
product  of  ERU  reliabilities. 

The  process  of  applying  ERUPT  is  essentially  a  working  backwards  from  observed 
component  (ERU)  reliability  to  estimates  of  human  reliability!  that  is,  human  reliability 
(probability  of  malfunction  detection  and  correction,  probability  of  malfunctions  induced 
during  maintenance)  is  derived  by  inference  between  actual  (shipboard  EF.U  reliabilities) 
and  inherent  (test)  reliabilities. 

The  Multiple  Correlation  Approach  to  maintenance  personnel  selection  is  similar  to 
ERUPT  in  that  it  uses  operational  and  test  data,  from  already  existing  systems  and/or 
components.  The  method  employs  data  (from  Navy  files  concerning  equipment  failure, 
e.g.,  number  of  repair  actions  required  per  unit  time  and  MTTR)  and  characteristics  of 
technicians  performing  the  maintenance  (age,  pay  grade,  experience,  time  left  in  service, 
education  rating  and  training  time).  Correlations  are  performed  using  personnel  charac¬ 
teristics  predictions  and  equipment  failure  data  as  criteria.  Multiple  correlations  are 
performed  and  interrelationships  between  the  variables  are  assessed.  Results  of  the 
analysis  may  be  used  to  identify  personnel  requirements  which  significantly  influence 
system  reliability  (in  terms  of  human  reliability  and  MTTR). 

Another  technique  reported  in  the  Human  Reliability  Prediction  Systems  User 
Manual  (1977)  is  Flow  Chart  (FC)  Maintainability  (M)  Prediction.  This  technique  is  one 
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which  emphasizes  troubleshooting  of  electronic  circuits,  and  its  purposes  are  to  estimate 
maintenance  times  for  circuits,  to  help  standardize  troubleshooting  procedures,  and  to  aid 
in  identifying  required  numbers  of  maintenance  steps  through  the  establishment  of 
troubleshooting  priorities. 

The  steps  required  to  appiy  the  technique  are  as  follows: 

•  Conduct  a  Level  of  Repair  (LOR)  analysis 

•  Conduct  a  Failure  Modes  Effects  Analysis  (FMEA)  to  determine 
symptoms  resulting  from  component  failures 

•  Evaluate  the  results  of  steps  one  and  two  above,  and  develop  a 
troubleshooting  flow  diagram 

The  diagram  should  indicate  logic  (from  step  two,  above)  and  repair  actions  that  may  be 
required  (from  step  one,  above).  The  diagram  should  be  developed  with  a  view  to 
(1)  employing  half -split  troubleshooting  techniques  (roughly  half  of  a  circuit  is  diagnosed 
as  faulty  or  not  at  each  test  point)  and  (2)  emphasizing  that  troubleshooting  actions  stem 
from  logical  deductions  (control  panel  indicators,  symptoms,  etc.).  The  final  step  in  the 
technique  is  to: 

•  Determine  or  estimate,  for  each  flow  chart  step  (logic  or  mainte¬ 
nance  action) 

-  test  equipment  setup  time 

-  overall  system  diagnosis  time 

-  modular  isolation /localization  time  (troubleshooting  to  mal¬ 
functioning  component  time) 

-  module  removal  time  (for  replaceable  components) 

-  module  installation  and  checkout  time 

Data  sources  are  quoted  as  tables  in  MIL  KDBK  472  (maintainability  prediction)  and 
maintainability  engineers  judgments.  Formulas  are  provided  to  determine  MTTR  (Mean 
Times  to  Repair)  using  the  data  from  MIL  HDBK  ^72  data. 

MONTE 

Another  technique  that  samples  end  points  of  decision  and  probability  stress  is 
called  Step  Through  Simulation  (Ulvila,  1977)  (Figure  2S),  which  uses  a  computer  program 
called  MONTE  (short  for  Monte  Carlo  Simulator).  Basically,  MONTE  uses  random 
sampling  techniques  for  estimating  outcomes  through  samples  of  paths  through  a  tree 
network,  yielding  a  probability  distribution  of  all  possible  outcomes.  A  decision  maker 
interacts  with  MONTE  via  a  CRT  and  light  pen.  The  program  poses  questions  and 
appropriate  data  to  the  decision  maker.  After  having  examined  the  data  provided,  the 
decision  is  entered  (e.g.,  engage  target).  The  random  sampler  selects  an  outcome  (based 
on  some  input  probabilities)  for  events  subsequent  to  the  decision  (target  destroyed. 
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target  missed,  damage  sustained,  etc).  Based  on  the  outcome,  another  decision  is 
requested  (resume  engagement,  return  to  base),  and  the  process  is  continued  until  the 
mission  (for  that  trial)  has  been  successfully  or  unsuccessfully  completed.  The  process  is 
iterated  several  times  until  sufficient  data  are  available  to  establish  an  outcome 
distribution. 

In  order  to  use  MONTE  it  is  necessary  to  construct  a  tree  network  that  shows  points 
where  decisions  are  to  be  made,  and  for  each  option  provided  in  a  decision,  estimations  of 
outcomes  to  a  subsequent  decision.  These  data  need  then  to  be  entered  into  the  program 
for  execution. 

Outputs  of  MONTE  include  frequency  distributions  of  mission  successes  and  failures 
and  decisions  made  at  various  points.  MONTE  could  be  a  valuable  tool  in  JPA  trade-offs, 
JPA  design,  identification  of  learning  objectives,  etc. 

Job  Performance  Aids 

Although  job  performance  aids  (JPA)  or  "performance  aiding  technology"  had  a 
definitely  slow  and  meager  period,  the  technology  is  being  rediscovered.  Previous 
dissatisfaction  with  development,  narrowness  of  research  and  ineffective  implementation 
has  been  overcome  by  the  need  for  more  effective  methods  of  using  the  level  of 
manpower  engendered  by  the  all-volunteer  military.  JPAs  are  considered  one  way  to 
enhance  human  performance  in  system  operation  and  maintenance. 

A  performance  aid  is  defined  as  that  which  stores  information  for  later  retrieval  in 
connection  with  the  performance  of  a  job  (Joyce,  1975).  It  facilitates  performance  by 
reducing  memory  requirements  imposed  on  the  user.  Examples  include  checklists, 
schematics,  assembled  procedures,  books  of  tables,  nomograms  and  technical  manuals. 
The  JPA  is  not  a  substitute  for  training,  its  basic  function  should  be  to  supplement  and 
support  training,  with  JPA  materials  being  used  in  actual  training  (Malone,  et  al,  1974). 
Their  basic  function  is  to  reduce  maintenance  costs,  increase  equipment  usability,  assure 
the  need  for  less  training  and  less  skilled  personnel.  The  use  of  JPA  significantly  reduces 
maintenance  costs  and  training  time. 

The  basic  decision  as  to  the  trade-off  between  training  and  use  of  JPAs  is  operator 
performance  as  well  as  savings  in  terms  of  cost  and  time.  Tabie  14,  adapted  from  the  Air 
Force  Systems  Command  DH  1-3  Handbook  on  Human  Engineering,  lists  the  factors  to  be 
considered  in  the  decision  as  to  training  vs.  JPAs.  Post  (1977)  developed  the  "warrants 
concept"  to  make  selection  of  formats  and  media  systematic,  match  cost  and  formats  to 
needs,  justify  costs  in  terms  of  maintenance  or  personnel  benefits,  improve  level  of  TM 
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TAELE  14 

TRAINING/ JPA  DECISION  CRITERIA 
(AFSC  DH  I  -3) 


Factors  in  Decision 


a.  Ease  of  learning 

b.  Ease  of  communication  by  book 

c.  Task  criticality 

d.  Tcsk  difficulty  (how  prone  to  inadequate  performance) 

e.  Importance  of  reaction  time  or  response  rate 

f.  Frequency  of  task  performance 

g.  Number  of  similar  tasks 

h.  Psychomoter  skill  component  of  tcsk 

i.  Rate  of  stimulus  input 

j.  Rate  of  response  output 

k.  Equipment  complexity 

l.  Rate  of  stimulus  input 

m.  Environmental  considerations 

n.  Mission  criticality 

o.  Consequences  of  improper  step  performance  on  task  performance 

p.  Personnel  hazards 

q.  Audience  career  orientation 

r.  Number  of  individuals  who  perform  a  task 


Put  in  Training 

Tasxs  that  are  not  very  easy  ro  learn  on-the-job 
Tcsks  that  are  hard  to  communicate  with  words 

c.  Tcsks  that  need  a  great  deal  of  practice  for  acceptable  performance  to  be 
estaDlished 

d.  Tasks  where  there  is  little  room  for  error 

e.  Tasks  where  consequences  of  error  are  serious 

f.  Tasks  that  do  not  take  exorbitant  sums  of  money  to  train 

g.  Tasks  whicn  are  performed  frequently  on-the-job 

h.  Tasks  in  which  the  required  speed  or  response  rate  does  not  permit  referring  to  a 
manual 

i.  Tasks  performed  by  a  large  proportion  of  the  individuals  in  a  given  specialty 


Put  in  Job  Performance  Aids 

a.  Behavior  sequences  that  are  long  and  complex 

b.  Tasks  that  are  rarely  performed 

c.  Tasks  that  involve  readings  and  tolerances 

d.  Tasks  that  can  be  mentally  rehearsed  before  the  need  to  perform  them  arises 

e.  Tasks  that  are  aided  by  the  presence  of  illustrations 

f.  Tasks  that  utilize  reference  information  such  as  tables,  graphs,  flow  charts  and 
schematics 

g.  Tasxs  with  branching  step  structures 
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usage  and  arrive  at  a  mix  of  aid  forms  in  consideration  of  tasks,  maintenance  and 
personnel  conditions.  This  approach  uses,  for  example,  the  maintenance  technical  data 
presentation  and  user  directive  aids  such  as  Fully  Proceduraiized  Job  Performance  Aids 
(FPJPA)  maintenance  action  procedures  in  a  technical  manual,  deductive  aids  such  as 
MDCs,  or  functional  flow  diagnosis.  This  approach  is  of  most  use  in  troubleshooting 
situations.  Post  (1977)  also  suggests  that  the  aid  have  two  features:  (1)  ability  to  be 
revised  by  user  feedback  or  conform  with  changes  in  the  systems  and  (2)  progress  with 
personnel  advances.  This  would  allow  the  user  to  progress  from  a  purely  directive  aid 
(with  complete  troubleshooting  procedures,  spelling  each  step  out)  to  a  deductive  aid  in 
which  the  user  selects  the  test  sequence  which  would  permit  him  to  deduce  the  cause  of  a 
problem.  The  deductive  aid  creates  in  the  user  a  sense  of  accomplishment;  however,  the 
personnel  organization  must  recognize  this  need  for  encouragement  of  the  user's  desire 
for  further  advancement.  Post  and  Price  (1976)  point  out  that  the  job  satisfaction  criteria 
must  be  considered,  i.e.,  opportunity  to  learn  the  meaningfulness  of  the  work  and 
challenge.  The  directive  aid  excels  in  this  criteria,  especially  for  experienced  techni¬ 
cians.  To  satisfy  this  situation,  the  hybrid  Augmented  Action  Tree  Troubleshooting  Aid 
(JPA  AATTA)  was  developed  which  allows  novice  technicians  to  conduct  troubleshooting 
while  at  the  same  time  giving  the  more  experienced  technicians  the  opportunity  to  learn 
career -relevant  skills. 

With  the  decision  made  as  to  the  use  of  JPAs,  various  JPAs  presently  conceived  can 
be  utilized  or  a  specific  JPA  developed.  One  of  the  most  comprehensive  discussions  of 
JPA  development  is  the  Air  Force  Human  Engineering  Handbook  DH  1-3.  This  document 
defines  specific  step-by-step  procedures  for  design  of  JPAs. 

Booher  (1977)  has  organized  a  modei  assuming  three  major  levels  for  JPA  technol¬ 
ogy;  (1)  the  JPA  system  levei,  (2)  the  performance  aid  component  level,  and  (3)  the 
performance  aiding  eiement  level.  The  system  is  comprised  of  several  types  of  formats 
for  different  categories  of  behavioral  tasks  (lubricate,  remove,  fault,  isolate,  etc.);  the 
formats  can  entail  tables,  lists,  functional  blocks,  matrices,  etc.  Use  of  a  task  analysis  or 
special  training  requirements  are  decided  at  this  step.  These  performance  aids  lead  to  the 
presentation  concept.  The  different  features  of  a  presentation  component  will  aid  a 
specific  behavior,  i.e.,  reading  voltages,  reading  wave  forms.  The  third  level,  performance 
aiding  element  level,  entails  decisions  as  to  readability  and  personnel  factors. 

One  available  technique  for  evaluating  JPAs  is  that  of  Ayoub,  et  ai  (1976).  The 
technique  presents  a  computerized  approach  for  developing  specific  rules  and  guidelines 
for  developing  and  producing  JPAs.  A  computerized  modei  of  the  maintenance  system 
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allows  evaluation  of  the  effects  of  different  management  policies  or  maintenance  tasks 
and  enables  performance  of  cost-benefit  analysis  of  design  alternations  and  approaches. 
It  also  allows  experimentation  with  different  JPA  alternatives  without  the  need  for  field 
investigations. 

FOMM 

Roader  and  Ranc  (1975)  state  that  the  Functionally  Oriented  Maintenance  Manual 
(FOMM):  (1)  decreased  maintenance  workload  by  improving  the  accuracy  of  maintenance 
actions;  and  (2)  reduced  costs  associated  with  maintenance  training,  i.e.,  textual  mate¬ 
rials.  The  production  of  such  materials  involves  understanding  of  system  operations,  as 
well  as  selecting  formats  readily  understood  by  the  user.  Neither  of  these  functions  are 
amenable  to  computer  applications.  Shriver  (1977)  states  that  there  is  a  definite  trend 
toward  automated  troubleshooting.  This  trend  is  based  on  the  realization  that  the 
troubleshooting  situation  is  not  a  problem-solving  situation  but  is  rather  an  operation 
which  can  be  fully  proceduralized.  There  is,  therefore,  a  good  deal  of  attention  being 
directed  toward  the  formal  and  informal,  due  to  the  lower  skill  levels  required  in  using 
FOMM  data.  FOMM  was  also  found,  however,  to  cost  40  percent  higher  than  conventional 
methods. 

TREES 

A  computerized  method  to  provide  maintenance  technicians  with  technical  data  is 
TREES  (Tree  Structured  Data).  TREES  also  provides  for  modifications  to  maintenance 
data  and  provides  taily  proceduralized  guidance  through  system  checkout  and  repair 
activities  (Colwell,  1971). 

The  method  employs  a  computer  program,  terminal  and  keyboard  interacting  with 
five  subprograms;  Build  (tree  construction),  Loads  (inputs  interaction  commands  and 
statements),  Edit  and  Bump  (data  base  maintenance),  and  Query.  The  Query  subroutine 
represents  the  interaction  of  the  maintenance  technician  in  the  course  of  maintenance 
activities.  The  program  provides  data  and  instructions  to  the  maintenance  technician,  and 
after  the  instructions  have  been  completed,  the  technician  responds  to  questions  posed  by 
the  subroutine  (multiple  choice,  yes  or  no  responses).  Based  on  the  response  input  by  the 
technician,  a  path  along  the  tree  is  selected  and  the  process  is  iteratea  until  troubleshoot¬ 
ing,  checkout  or  repair  is  completed. 

Instructional  Systems  Development 

Instructional  Systems  Development  (ISD)  is  a  methodology  for  managing  training 
system  design  and  development.  ISD  divides  training  system  development  into  distinct 
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phases  with  steps  and  objectives  to  accomplish  within  each  (Funaro,  1978).  An  ISD  model 
of  particular  interest  is  the  Interservice  Procedures  for  Instructional  Systems  Develop¬ 
ment  (IPISD'',  reported  by  Logan  (1977).  IPISD  model  is  divided  into  five  phases: 

Analysis 

2.  Design 

3.  Development 

'4.  Implementation 
5.  Control 

Activities  and  outputs  for  each  of  these  phases  are  presented  in  Table  15.  As  a 
management  tool,  IPISD  structures  training  system  development  according  to  the  phase 
activities  outlined  in  the  table.  Analytic  and  design  techniques  used  to  accomplish  the 
objectives  of  each  step  are  more  or  less  free  to  vary  according  to  such  considerations  as 
system  size  and  complexity,  whether  or  not  the  training  system  is  for  an  existing  or  a  new 
weapon  system,  whether  the  training  system  is  designed  towards  system  operation, 
maintenance  or  both  and  so  on. 

Training  Requirements  Analysis 

Training  Requirements  Analyses  is  a  tool  that  identifies  required  skills  and  knowl¬ 
edges  of  system  operators  and  maintained.  Job  tasks,  training  tasks,  performance 
standards,  and  central  skills  are  identified  for  each  position.  The  analysis  is  basically  an 
examination  of  position  description,  task  analysis,  JPA  requirements,  and  IL5  (LOR,  for 
example)  data,  and  the  appropriate  skill  and  knowledge  requirements  are  identified. 

Job  Analysis 

Job  Analysis  is  a  method  for  identifying  and  analyzing  training  requirements  from 
data  available  in  systems  similar  to  the  developing  system.  Like  Training  Requirements 
Analysis,  Job  Analysis  identifies  and  catalogs  job  tasks  according  to  function  and  skill  and 
knowledge  requirements  and,  as  its  primary  purpose,  aids  in  identifying  training  objectives 
and  provides  input  to  training  system  trade-offs  (JPA,  media  selection,  personnel 
requirements,  etc.). 

Training  Media  Selection 

Training  media  selection  represents  a  central  and  difficult  step  in  a  training  system 
development  process.  Learning  objectives  must  be  matched  to  media  types  and  methods, 
within  such  constraints  as  cost,  training  time  and  manpower  availability. 
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TABLE  IS 

PHASE  ACT'-.  r:ES  AND  OUTPUTS  FOR  THE  1PISD  MODEL 

(LOGAN,  1977) 


Phase 

Activities 

OutDUtS 

Analyze 

• 

Job  analyses 

Job  task  list 

• 

Task  function  selection 

- 

Training  task  list 

• 

Select  job  performance 

- 

Job  performance  measures 

measures 

- 

Instructional  setting 

• 

Analyze  existing  courses 

selection 

Design 

• 

Develop  learning  objectives 

. 

Learning  objectives  and 

• 

Develop  tests 

analyses  of  each  task 

• 

Describe  entry  behaviors  of 

- 

Test  items  selected 

students 

- 

Entry  behevior  test 

• 

Develop  task  learning  struc¬ 
ture 

Dependent  task  sequences 

Develop 

• 

Learning  events/activities 

Learning  objective  tax¬ 

specification 

onomies 

• 

Plan  instruction  management 

- 

Learning  materials 

• 

Select  materials 

- 

Instructions  for  all 

• 

Develop  instructions 

learning  objectives 

• 

Instruction  validation 

Tested  and  revised  in¬ 
structional  materials 

Implement 

• 

Apply  instructional  manage¬ 

Requirea  management  docu 

ment  plan 

ments  and  tools 

• 

Conduct  instruction 

Implementation  of  training 
system 

Control 

• 

Conduct  internal/externai 

Instructional  effectiveness 

evaluation 

data 

• 

Analyze  end  revise  system 

* 

Field  job  performance  data 

- 

Instructional  system  re- 

vision. 
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A  technique  known  as  Means  to  Achieve  Performance  (MAP)  (Tannebaum,  1976)  was 
developed  to  aid  in  performing  trade-offs  between  (1)  personnel  selection,  (2)  training,  (3) 
performance  aids  and  (4)  reference  documents  in  order  to  optimize  human  performance. 

Application  of  MAP  employs  a  checklist  showing  pertinent  tasks  and  activities  and 
each  of  the  options  stated  above.  Tasks  are  assigned  to  an  option  (or  phase)  according  to 
guidelines  provided  in  Table  16. 

A  technique  for  selecting  cost-effective  instructional  media,  TECEP  (Training 
Effectiveness,  Cost  Effectiveness,  Prediction  Techniques),  has  been  developed  by  the 
Training  Analysis  and  Evaluation  Group  at  the  Naval  Training  Equipment  Center  (Braby, 
Henry,  and  Morris,  1974).  The  purpose  of  TECEP  is  to  aid  in  choosing  cost-effective 
instructional  media  for  training  systems. 

The  technique  is  applied  in  nine  steps  which  organize  training  objectives  into  groups; 
potential  learning  strategies  for  each  group  are  defined,  appropriate  media  are  selected 
and  cost  trade-offs  are  performed.  The  specific  steps  are  as  follows: 

1.  Classify  training  objectives  according  to  the  sixteen  categories  provided 
(continuous  movement,  decision-making,  voice  monitoring,  etc.). 

2.  For  each  category,  define  a  learning  strategy.  A  summary  table  of 
learning  guidelines  for  each  category  is  provided;  for  example,  learning 
guidelines  for  decision-making  tasks  include: 

-  access  to  relevant  data  provided  to  trainees 

-  overlearning  of  skill  required  to  help  overcome  effects  of  stress, 
etc. 

3.  Identify  media  characteristics  which  match  learning  strategy  and  objec¬ 
tives.  The  report  provides  general  training  media  characteristics  ac¬ 
cording  to  five  classes: 

-  stimulus  capabilities 

-  trainee  response  modes 

-  information  feedback  logic 

-  event  sequence  logic,  and 
instructional  setting 

4.  Select  media  that  contain  the  characteristics  identified  in  step  three.  A 
taoie  of  media  classes  is  provided  (and  presented  in  Table  17). 

5.  Reject  inappropriate  or  impractical  media  approaches  according  to 
considerations  such  as: 

-  state-of-the-art  of  the  medium 

-  system  size  and  inherent  medium  practicality 
time  requirements 

6.  For  each  remaining  media,  estimate  time  to  achieve  objectives. 

7.  Propose  alternate  training  systems  for  trade-offs. 

8.  Estimate  cost  (annual)  for  each  training  system. 

9.  Select  optimum  training  media  mode. 
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TABLE  16 

GUIDELINES  FOR  SELECTION  OF  MEANS  TO  ACHIEVE  PERFORMANCE 

(from  Tannenbaum,  1976) 


PERSONNEL  SELECTION  may  be  an  option 

when: 

1.  Availabe  Beil  System  people  have 
the  required  skills  and  knowledge. 

2.  Skills  and  knowledge  are  complex, 
and  training,  references  or  perfor¬ 
mance  aids  are  costly  to  develop 
and  maintain. 

3.  Not  many  people  are  required  to 
perform  this  activity. 

4.  Rapid  start-up  time  is  required. 


TRAINING  may  be  an  option  when: 

1.  Speed  and/or  accuracy  levels  must 
be  demonstrated  before  starting  on 
the  job. 

2.  On-the-job  speed  is  so  critical  that 
there  is  not  time  to  use  a  perfor¬ 
mance  aid  or  a  reference  document. 

3.  A  period  of  familiarization  is  needed 
before  starting  work. 

4.  Activities  may  be  too  complex  to 
learn  without  instructions. 

5.  Skills  and  knowledge  required  are 
system  specific. 

6.  Performance  aids  and/or  references 
need  practice,  demonstration,  or 
explanation  before  use  on-the-job. 

7.  Large  numbers  of  performers 

8.  Training  modules  exist  for  this 
activity. 


Continued 

9.  It  is  more  cost  effective  to 
train  a  lower  paid  person  than 
to  select  existing  personnel. 

10.  Skills  and  knowledges  need  to 
be  enhanced. 


PERFORMANCE  AIDS  may  be  an  option 

when: 

1.  Speed  and/or  accuracy  must  be 
assured  at  the  time  of  perfor¬ 
mance. 

2.  Procedures  or  sequences  are 
required  and  memory  needs  to 
be  supplemented  or  substituted. 

3.  Assistance  is  needed  to  make 
decisions  and/or  juagments. 

4.  The  speed  of  finding,  retriev¬ 
ing,  or  using  information 
needs  to  be  increased. 

5.  Activities  are  considerea  to 
be  complex. 

6.  Activities  are  performed  in¬ 
frequently. 

7.  Conversion  of  information  is 
required. 

8.  Quantity  of  information  :s  too 
great  to  remember. 

REFERENCE  DOCUMENTS  may  be  an  option 

when: 

1.  Activities  are  performed  infre¬ 
quently;  retention  of  information 
is  not  likely. 
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TABLE  16 
(Continued) 


REFERENCE  DOCUMENTS  may  be  an  option 
when: 


2.  Decision-making  guidance  is  required; 
the  worker  will  need  information 
ordinarily  presented  in  handbooks, 
extensive  formula  tables,  computer 
programs,  and  the  like,  rather  than 
performance  aids. 

3.  A  full  description  of  the  work  is  re¬ 
quired,  possibly  including  background 
or  perspective,  in  order  to  plan  or 

to  carry  out  the  instructions. 


4.  Visual  reinforcement  for  procedures, 
such  as  that  provided  by  illustrations, 
or  other  graphics,  is  required. 

5.  Cross-referencing  is  required,  such 
as  matching  information  from  one 
complicated  source  with  information 
from  another  similar  source. 
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TABLE  17 
MEDIA  CLASSES 

(from  Braby,  Henry  and  Morris,  197*0 


Audio  Disc  Playback  System 

Audio  Tape  System 

Dial  Access  Information  Retrieval 
System  -  Random  Audio 

Dial  Access  Information  Retrieval 
System  -  Scheduled  Audio 

Language  Laboratory  -  Audio,  Active  - 
Compare  Moae 

Language  Laboratory  -  Audio  Passive 
Moae 

Physiological  Trainer  (Hostile 
Environment)  Auditory 

Radio  System  -  AM/FM 

Radio  System  with  Responders 

Telephone  Conference  System 

Carrel,  AV  Equipped 

Computer  Assisted  Instruction  (CAI) 

Computer  Assisted  Instruction:  IBM 
Aids 

Computer  Assisted  Instruction:  Plato 
IV  Basic  Configuration  and  Audio 

Dial  Access  Information  Retrieval 
System  Scheduled  Audio/Video 

Filmstrip  Projection  System  with  Audio 

Game,  Manual  Non-Simulation 

Game,  Manual  Simulation 

Motion  Picture  Projection  System  - 
Commercial,  loMM  and  Super  8MM  Films 

Motion  Picture  Projection  System  - 
Low  Budget  I6MM  and  Super  3MM  Films 

Stuaent  Response  System:  AV  Supported 


Teaching  Machine,  Breaching,  Still 
and  Motion  Visual  with  Audio 

Teaching  Machine,  Linear,  Still  and 
Motion  Visual  with  Audio 

Teleconference  System 

Television  -  Cable  (CATV) 

Television  -  Cartridge  (CTV) 

Television  -  CCTV  with  Feedback 

Television  -  Closed  Circuit  (CCTV) 
without  Feedback 

Television  -  Open  Broadcast 

Television  -  Portable  Video  Tape 
System 

Television  -  Video  Disc  System 

Carrel,  Laboratory 

Computer  Assisted  Instruction:  IBM 
Aids  with  Adjunct  Equipment 

Computer  Assisted  Instruction:  Plato 
IV,  Basic  Configuration  with  Adjunct 
Equipment 

Computer  Assisted  Instruction:  Plato 
IV  Basic  Configuration  with  Adjunct 
Equipment  and  Audio 

Filmstrip  Projection  System  with 
Audio  and  Adjunct  Equipment 

Operational  Equipment  with  Manuals 

Operational  System  -  Real  Environment 

Operational  System  -  Synthetically 
Simulated 

Operational  System  -  Synthetically 
Stimulated 

Procedure  Trainer 


TABLE  17 
(Continued) 


Procedure  Trainer  -  Adjunct  Displays 
and  Logics 

Simulator 

Simulator  -  Adjunct  Displays  and 
Logics 

Teaching  Machine  -  Branching,  with 
Adjunct  Equipment 

Television  -  Video  Disc  with  Adjunct 
Equipment 

Audio  Tape  with  Printed  Material 

Classroom  -  Traditional 

Microfilm  with  Information  Mapping 
and  Audio 

Multi-Media  Kits  with  Instructor 

Overhecd  Projection  System  with 
Instructor 

Sound  Slide  Projection  System 

Teaching  Machine  -  branching.  Still 
Visual  with  Audio 

Teaching  Machine  -  Linear,  Still 
Visual  with  Audio 

Multi-Media  Kits  for  Trainees 

Sound  Slide  Projection  System  with 
Adjunct  Equipment 

Game  -  Computer  Supported  Simulation 

Models  and  Static  Mockups  -  Small 
Scale 

Physiological  Trainer  (Hostile 
Environment)  Visual 

Computer  Assisted  Instruction:  Plato 
IV  Basic  Configuration 

Filmstrip  Projection  System 


Logic  Trainers 

Microform  with  Information  Mapping 
and  Adjunct  Equipment 

Mockups,  Panels,  and  Demonstrators  - 
Dynamic 

Programmed  Text  -  Branching  with 
Adjunct  Material/Equipment 

Automatic  Raters  -  Informal  Training 

Carrel  -  Dry 

Case  Study  Folder 

Flash  Cards 

Microform  with  Information  Mapping 

Printed  Materials  -  Handouts 

Printed  Materials  -  Performance  Aids 

Printed  Materials  -  Reference  Books 

Printed  Materials  -  Reference  Charts 

Printed  Materials  -  Self  Scoring 
Exercises 

Printed  Material  -  Textbook 
Printed  Material  -  Workbook 
Programmed  Text  -  Branching 
Programmed  Text  -  Linear 
Simulation  -  Paper 
Slide  Projector  System  -  2"  x  2" 

Study  Card  Sets 

Teaching  Machine  -  Branching,  Still 
Visual 

Teaching  Machine  -  Linear,  Still,  Visual 
Do-It-Yourself  Kits 
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TABLE  17 
(Continued) 


Mockups,  Panels  and  Demonstrators  - 
Static 

Specimen  Sets 

Computer  Simulation  -  Off-Line 
Computer  Simulation  -  On-Line 


Game  -  Computer  Simulation  -  Solitaire  - 
with  Visual  Display 

Physiological  Trainer  (Hostile  Environ¬ 
ment)  Surface  and  Internal  Senses 
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Campbell  and  Hughes  (1978)  have  developed  an  11-step  process  to  help  determine 
media  requirements  for  each  learning  objective  and  to  optimize  mixes  of  media  applicable 
to  those  requirements.  The  11  steps  of  the  technique  (see  Figure  29)  are  devoted  to 
determining  and  analyzing  (1)  study  session  and  (2)  training  device  media  alternatives. 
The  1 1  steps  are  as  follows: 

1.  Identify  learning  objectives  as  being  study  type  (classroom,  i.e.)  or  training 
device  type  (maintenance  trainers,  etc.)  objectives 

2.  Design  study  session  objectives  as  applicable  to,  or  requiring  example  sets 
(troubleshooting  input,  etc.),  or  not  requiring  examples  (fixed  procedures  for 
example) 

3.  Identify  special  requirements  for  study  session  objectives.  Examples  include 
color,  cues,  instructor  evaluation,  etc. 

4.  List  acceptable  media  for  each  study  session  objective.  The  others  provide  a 
matrix  with  special  characteristics  listed  in  the  column  and  media  types  along 
the  row  (CAI,  lecture  and  slides,  etc.),  and  appropriate  media  are  identified 
for  special  characteristics. 

5.  Rank  applicable  media  according  to  estimated  cost,  present  media  availability, 
etc. 

6.  Select  study  session  media  mixes 

7.  Identify  number  of  learning  objectives  in  each  medium 

Steps  2  through  7  deal  with  study  session  media  issues;  the  remaining  steps,  S 
through  1 1 ,  deal  with  device  media  selection. 

8.  Identify  device  session  participation  requirements,  e.g.,  displays,  mock- 
ups,  schematic  representations,  etc. 

9.  Classify  objectives  according  to  equipment  (training  and  actual)  compat¬ 
ibility  requirements 

10.  Group  objectives  to  arrive  at  an  optimal  set  of  training  descriptions. 

11.  Determine  final  trainer  configuration  by  assigning  learning  objectives  to 
training  devices 

Post,  Price  and  Diffley  (1976)  have  developed  a  tool  which  aids  in  the  selection  of 
formats  and  media  for  presenting  maintenance  information  (Post-Price  Method).  The 
method  was  developed  with  a  view  to  implementation  in  the  early  stages  of  system 
development  attempts  to  match  technical  manual  formats  with  personnel,  equipment  and 
workspace  characteristics. 

The  method  is  applied  in  five  steps: 

1.  Identify  maintenance  actions  and  system  condition  dat3 

2.  Identify  areas  where  standard  formats  are  required  (Standard  Operating 
Procedures,  maintenance  complexity,  time  criticality,  etc.) 

3.  Select  formats  for  troubleshooting  actions 
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4.  Select  formats  for  removai/repiacement  actions 

5.  Establish  Technical  Manual  (TM)  support  requirements 

The  first  step  requires  the  completion  of  a  Task  Identification  Matrix  (Figure  30),  which 
identifies  the  types  of  maintenance  actions  required  for  given  components  being  sup¬ 
ported.  A  procedure  is  provided  for  the  selection  of  formats  based  on  cost,  available 
formats,  system  computations  (troubleshooting  complexity,  personnel  turnover,  etc.).  A 
guide  to,  and  recommendations  for,  Technical  Manual  Support  requirements  are  provided. 
Three  issues  of  TM  support  requirements  are  identified  as  being  (1)  ease  of  access, 
(2)  ease  of  storage,  distribution  and  updating,  and  (3)  useabiiity  over  extended  use  or 
adverse  environments. 

The  method  is  applied  according  to  the  following  considerations: 

•  Number  of  maintenance  actions  and  conditions  under  which  they  are 
performed 

•  System  considerations  such  as  complexity,  size,  workspace  and  main- 
tainer  characteristics 

•  Combination  of  maintenance  actions  as  homogeneous  sets,  to  reduce 
TM  complexity  and  developmental  time  and  costs 

•  Innovative  formats  and  media  to  best  suit  a  particular  system  and  its 
conditions 

•  TM  preparation  cost 
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FIGURE  30 


TASK  IDENTIFICATION  MATRIX  (TIM) 
ADAPTED  FROM  POST,  PRICE  &  DIFFLEY  (1976) 
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3.3  HFE  Technologies  Applicable  During  Full-Scale  Engineering  Development  Phase 

The  major  human  engineering  efforts  during  this  phase  consists  of  evaluating  design 
concepts  generated  during  the  Demonstration/Vaiidation  phase,  limited  redesign  of 
concepts  and  determination  of  HFE  design  criteria  and  procedures.  Most  of  the 
technologies  applicable  during  this  phase  have  been  implemented  during  the  earlier 
acquisition  phases,  e.g.,  IPISD,  Task  Analysis,  HEDGE,  HFTEMAN,  CAFES,  MIL  HDBK 
472,  etc. 

Task  Equipment  Analysis 

Task  Equipment.  Analysis  (TEA)  is  applied  to  describe  and  analyze  tasks  demon¬ 
strating  how  an  operator/maintainer  interacts  with  actual  equipment.  A  TEA  format  is 
presented  in  Figure  31.  The  analyses  typically  lists: 

•  Tasks  to  be  pe^rmed 

•  Associated  controls/locators 

•  Method  of  control  activation 

•  System  responses 

•  System  response  time 

•  Associated  displays/locators 

•  Display  indicators 

•  Job  aids/tools/test  equipment  (for  maintenance  TEA) 

TEA  can  serve  as  a  means  of  ensjring  that  ail  operational  and  performance  requirements 
asr-'ciated  with  the  equipment  are  satisfied  by  the  tasks. 

Timeline  Analysis 

Timeline  Analysis  provides  indications  of  temporal  relationships  among  tasks  and 
also  indicates  the  duration  of  individual  tasks.  The  technique  is  relatively  easy  to  apply 
and  can  be  very  useful  in  identifying  high  and  low  operator  load  at  various  points  in  a  task 
sequence.  In  application,  tasks  are  sequentially  listed  (on  a  formal  timeline  sheet)  in  a 
column  form,  task  duration  time  estimates  are  indicated  by  a  bar  graph,  at  the 
appropriate  task  initiation  and  termination  points  along  the  time  (X-axis)  dimension.  At 
any  point  of  time  during  a  task  sequence  for  an  operator,  the  analysis  can  indicate: 

•  Number  of  concurrent  tasks 

•  Rapidity  of  task  performance,  and 

•  Operator  overload. 

The  analysis  can  be  applied  for  any  level  of  detail  required,  e.g.,  gross  tasks  such  as 
"monitor,"  "verify,"  etc.,  or  refined  task  elements  demonstrating  task  completion  medium, 
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e.g.,  "right  hand-depress  'launch'."  The  mere  general  task  ievei  is  usually  selected; 
however,  since  the  detailed  task  element  timeline  is  both  difficult  to  develop  and 
interpret.  Examples  of  timelines  are  presented  in  Figures  32  through  3^. 

Workload  Estimation 

Workload  estimation  techniques  and  analyses  that  are  applicable  during  the  Full- 
Scale  Engineering  Development  Phase  differs  greatly  from  the  techniques  previously 
reported.  The  techniques  reported  earlier  (such  as  SW  models,  CAFES  WAM)  rely  on 
judgments,  task  analysis  to  predict  operator  workload  interacting  with  evolving  systems. 
The  present  techniques  measure  (to  some  extent)  workload  of  an  operator  interacting  with 
hardware. 

Wierwille  and  Williges  C197S)  have  recently  surveyed  and  analyzed  operator  and 
workload  assessment  techniques.  Twenty-eight  separate  techniques  were  categorized  into 
four  separate  groups.  These  four  groups  are; 

•  Subjective  opinions 

•  Spare  mental  capacity  methods 

•  Primary  task  measures,  and 

•  Phsysiological  measures 

For  each  of  the  28  techniques,  theory,  background,  methods  and  apparatus  and  areas  of 
application  are  discussed.  Subjective  opinions  as  a  means  to  measure  or  estimate 
workload  entails  the  use  of  rating  scales,  structured  questionnaires,  interviews,  etc. 
Spare  mental  capacity  methods  vary  somewhat  in  nature,  but  generally  they  require  that  a 
task  be  exercised  while  operational  tasks  are  being  executed,  i.e.,  an  additional  task  not 
related  to  system  operation  is  performed.  The  assumption  is  that  if  operational  task  load 
is  high,  the  additional  task  cannot  be  completed  successfully,  if  at  all.  These  additional 
tasks  can  be  such  things  as  time  estimation,  tracking,  etc.  Primary  task  measures 
attempt  to  measure  mental  workload  as  a  function  of  primary  task  performance.  A 
change  in  operator  performance  indicates  an  increase  in  mental  workload.  Physic :  gical 
measures  use  physiological  responses  of  an  operator  as  a  measure  of  workload.  As 
workload  changes,  bodily  responses  are  deemed  to  change  also.  Physiological  measures 
can  be  used  singly  or  in  combination.  Examples  of  physiological  measures  are;  respira¬ 
tory  activity,  circulatory  activity,  galvanic  skin  responses,  EEG  data.  etc. 

3.4  HFE  Technology  Assessment 

The  HFE  technology  assessment  reported  here  is  concerned  with  the  degree  to  which 
the  outputs  of  a  particular  HFE  method  or  technique  satisfy  Human  Factors  Engineering 
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requirements  shown  in  Figure  1.  The  assessment  is  not  concerned  with  the  procedure  by 
which  data  are  generated,  synthesized  or  analyzed  in  order  to  achieve  those  outputs. 

Tabie  IS  and  Figure  35  achieve  the  purposes  of  this  technology  assessment.  Table 
IS  describes,  for  each  technology: 

•  Technology  type 

-  descriptive 

-  analytic 

-  design 

-  evaluation 

-  diagnostic 
integrative,  and/or 

-  data 

•  Application 

•  Inputs  and  outputs 

•  Cost 

•  Impact  on  system  design 

•  Alternative  technologies 

•  Source 

•  Special  notes 

The  technology  type  entry  in  Table  13  refers  to  the  overall  objectives  of  the 
technology,  description,  analysis,  etc.  The  distinction  between  evaluative  and  diagnostic 
technologies  is  made  since  all  evaluation  technologies  do  not  diagnose  problem  areas 
within  a  design  scheme,  but  rather  only  evaluate  the  total  design  as  acceptable  or  not 
acceptable.  Application  refers  to  the  method(s)  by  which  the  technology  is  applied,  e.g., 
computer,  hand  calculations,  drawings,  etc.  Cost  is  considered  to  be  high  (H),  medium  (M) 
or  low  (L),  depending  on  method  of  application  and  complexity  of  the  technique.  Cost  is, 
however,  difficult  to  estimate,  particularly  with  the  computerized  techniques.  Invoking  a 
computer  simulation  for  a  very  small  or  simple  system  will  obviously  be  more  expensive 
than  hand  techniques,  but  many  manual  techniques  as  applied  to  complex  systems  with 
highly  involved  human  operations  are  similarly  unfeasible.  At  what  point  certain  manual 
techniques  become  less  cost  beneficial  than  the  computerized  methods  cannot  be 
determined;  however,  since  this  paper  deals  with  major  weapon  systems,  cost  considera¬ 
tions  may  begin  to  favor  the  computerized  techniques,  thereby  rendering  cost  judgment 
tenable.  Impact  on  system  design,  like  cost,  is  judged  to  be  high,  medium  or  low, 
depending  on  the  manner  and  frequency  which  output  data  of  a  technique  or  method  is 
used  to  satisfy  HFE  requirements.  Alternative  technologies  are  those  which  may  be  used 
in  lieu  of  the  particular  method  being  assessed,  e.g.,  have  similar  outputs,  purposes,  goals, 
etc.  Alternative  technologies  may  vary  in  terms  of  cost,  system-specific  applicability, 
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and  so  on.  Therefore,  alternate  technologies  are  in  no  way  considered  in  this  report  as 
"inter  chan  geable. " 

Figure  35  shows,  for  each  technology,  its  applicability  to  specific  HFE  requirements 
shown  in  Figure  1. 

Specific  techniques  are  Listed  in  the  column;  HFE  requirements  are  the  row  entries. 
Where  a  column  entry  meets  a  row  entry  the  following  apply: 

•  No  entry  indicates  that  there  is  no  relationship  between  the  technol¬ 
ogy  and  requirement 

•  A  indicates  that  the  column  technique  is  applicable  to  the  row 
requirement 

•  A  "A"  entry  indicates  that  the  row  entry  is  input  to  the  column 
technique. 
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4.0  HFE  TECHNOLOGY  SHORTFALLS 

In  this  section,  HFE  technological  shortfalls  relevant  to  the  following  are  discussed: 

•  Similar  systems  analysis 

•  Special  environments 

•  Design  criteria/military  specifications  and  standards 

•  Complex  workspace  design 

•  Maintainability  design  and  evaluation 

•  Test  and  evaluation 

•  Training  system  design 

4.1  Similar  System  Analysis 

Similar  Systems  Analysis  represents  an  HFE  requirement  that  has  not  experienced 
much  intense  interest  in  HFE  R<5cD.  The  need  for  a  proceduralized  method  of  analyzing 
similar  systems  and  subsystems  has  increased  over  the  years  since  the  development  of 
computerized  human  and  human-system  simulators  which  rely  on  human  and  system 
reliability  and  time  input  data.  These  data,  more  often  than  not,  are  not  available  in 
either  data  bases  or  research  literature.  The  result  is  that  many  subjective  judgments  on 
the  part  of  HF  specialists  and  hardware  engineers  are  required  to  implement  these  man- 
machine  models.  Apart  from  the  obvious  data  available  for  analysis  of  similar  systems 
(manning  levels,  functional  allocations,  task  sequences,  etc.),  detailed  information  may  be 
acquired,  e.g.: 

•  iMission  segment  times,  timelines 

•  Task  sequence  duration  time 

•  System,  subsystem  and  component  reliabilities 

•  Operator  reliabilities 

•  Operator  task  loading 

•  Task  execution  times 

•  HFE  design  deficiencies 

•  Subsystem  and  component  response  times,  etc. 

A  variety  of  tools  exist  for  collecting  such  data,  but  are  typically  designed  for 
shipboard  data  collection,  and  many  of  these  are  directed  towards  CIC  (Combat 
Information  Center)  operations.  Notables  in  this  group  include: 

•  OPREDS  (Operational  Performance  Recording  and  Evaluation  Data 
System) 

•  ADER  (Automatic  Data  Extraction  and  Reduction) 
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•  OPMS  (Operator  Performance  Measurement  System,  directed  towards 
radar  tracking  task  performance  measurement) 

The  basic  method  of  operation  of  these  techniques  is  the  recording  of  controi  activations 
and  display  information  for  all  controls  and  displays  over  time.  The  recordings  are  taken 
from  the  equipment  directly,  i.e.,  hardwired  to  controls  and  displays.  The  recordings  can 
e  piayed  back  to  reconstruct  actual  operations,  noting  operational  times,  equipment 
activations,  errors,  etc.,  in  a  nonobtrusive  fashion. 

The  chief  difficulty  in  acquiring  data  with  hardware  such  as  this  for  certain  major 
weapons  systems  (e.g.,  small  aircraft)  is  the  availability  of  space  for  equipment  storage. 

FLAG  is  a  computerized  data  base  of  HE  design  deficiencies  reported  (via 
discrepancy  sheets)  for  the  P-3B/C,  SH-2,  S-3A,  F-14,  and  F-18  aircraft  (F-18  aircraft 
data  are  to  be  input  as  the  data  become  available).  FLAG  is  to  be  continually  updated  so 
as  to  be  an  aid  in  identifying  HFE  design  problem  areas  for  future  aircraft  acquisitions. 

Specific  HFE  technology  shortfalls  relevant  to  similar  systems  analysis  are: 

•  There  is  no  general  tool  for  collecting  quantitative  operational  data 
from  similar  systems 

•  Beyond  FLAG,  which  is  directed  solely  towards  identifying  aircraft 
human  engineering  deficiencies,  there  is  no  readily  available  HFE 
data  base  concerning  reported  HFE  design  deficiencies  inherent  in 
specific  systems  or  system  classes. 

•  There  is  no  proceduraiized  method  (other  than  formal  HFE  TJcE)  to 
analyze  similar  systems  in  order  to  help  identify  developing  system 
HFE  design  issues 

Recommendations:  based  on  the  identified  technology  gaps  relevant  to  similar  systems 
analysis,  the  following  recommendations  are  made: 

•  Determine  the  feasibility  of  modifying  operational  recording  systems 
(such  as  OPREDS)  such  that  they  may  offer  a  mors  universal 
applicability 

•  Determine  the  feasibility  of  establishing  an  HFE  design  deficiency 
report/retrieval  system,  similar  to  FLAG,  for  other  classes  of  sys¬ 
tems 

•  Examine  the  potential  and  feasibility  of  a  similar  systems  and 
components  evaluation  methodology  to  assist  in  identifying  potential 
design  issues  in  developing  systems.  The  methodology  should  address 
such  areas  as  operability,  maintainability,  manning  and  training, 
system/mission  reliabilities  and  component  reliabilities 

4.2  Special  Environments 

With  the  deveiooment  of  new  technologies  being  incorporated  into  weapon  systems, 
the  human  operator  has  been  subjected  to  different  and  more  demanding  work  environ- 
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merits.  These  technologies  both  introduce  operators  to  these  environments  (space 
systems,  high  altitude  flight,  arctic  environments,  etc.)  and  subject  operators  to  special 
environments  (high  acceleration  aircraft,  high  vibration  shipboard  spaces  and  ships, 
radiation,  etc.).  Within  the  context  of  special  environments,  HFE  issues  such  as 
performance  degradation,  environmental  design,  safety  and  personnel  issues  are  ad¬ 
dressed.  For  large,  sophisticated  systems,  experimental  research  will  be  implemented  to 
determine  performance  degradation,  to  evaluate  environmental  design,  etc.  For  systems 
where  such  research  cannot  be  justified  or  funded,  HE  methods  and  data  will  be  relied 
upon  to  assist  in  developing  environmental  design  concepts.  Within  this  context,  the 
following  technology  gaps  are  identified: 

•  A  sufficient  HFE  data  base  relating  to  performance  degradation  as  a 
function  of  environmental  dimensions  is  not  available. 

•  Personnel  selection  techniques  for  special  environments  are  scattered 
and  insufficient  in  terms  of  addressing  a  broad  spectrum  of  environ¬ 
ments. 

•  Anthropometric  design  methodologies  (apart  from  aircraft  stations) 
generally  do  not  exist,  particularly  for  such  conditions  as  cold 
weather  maintenance,  accessibility  design,  etc. 

Recommendations:  specific  recommendations  to  be  made  are  as  follows: 

•  While  a  good  deal  of  research  related  to  performance  degradation  in 
special  environments  has  been  (and  is  being)  performed,  it  has  not 
been  reviewed,  abstracted  and  presented  in  usable  form.  Develop¬ 
ment  of  an  environmental  design  handbook  for  Navy  systems,  that 
encompasses  somewhat  special  environments,  is  recommended. 

•  Many  psychometric  selection  tools  exist;  however,  general  purpose 
test  batteries  directed  at  selecting  personnel  for  special  environ¬ 
ments  do  not.  It  is  recommended  that  such  tools  be  developed. 

•  Computerized  modelling  of  the  anthropometry  of  military  personnel 
have  been  developed  (CAFES,  CGE,  COMBIMAN,  CAPE,  etc.).  The 
thrust  of  hese  models  has  been  the  simulation  of  aircraft  stations.  It 
is  recommended  that  the  feasibility  of  modifying  some  of  these 
models  in  order  to  extend  applicability  to  other  system  types  be 
determined. 

Kennedy  (197S)  has  reported  a  program,  at  the  Naval  Aerospace  Medical  Research 
Laboratory  detachment,  New  Orleans,  and  the  Pacific  Missile  Test  Center,  known  as 
PETER  (Performance  Evaluation  Test  for  Environmental  Research).  The  purpose  of  the 
program  is  to  develop  a  human  performance  test  battery  for  personnel  selection  and 
performance  prediction  in  special  environments. 
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4.3  HFE  Design  Criteria  and  Military  Specifications  and  Standards 


The  following  shortfalls  have  been  identified  regarding  HFE  design  criteria  and 
miiitary  specifications  and  standards: 

•  For  many  systems,  due  to  space  limitations,  environments,  cost 
constraints,  etc.,  existing  criteria  and  standards  are  not  feasible  and 
cannot  be  implemented. 

•  Research  data  often  are  conflicting  with  standards  and  criteria. 

•  Anthropometric  standards  often  exclude  a  large  proportion  of  the 
available  manpower  pool. 

•  Where  special  circumstances  do  not  prohibit  compliance  with  stan¬ 
dards  and  specifications,  they  are  inadequate  in  terms  of  addressing 
maintainability  design,  anthropometry,  component  selection  and 
functional  allocations,  as  a  function  of  cost,  relative  reliabilities  and 
availability. 

•  Military  standards  and  specifications  relating  to,  or  affecting,  HFE 
design  are  scattered  throughout  Human  Engineering  standards  and 
specifications,  NATO  agreements,  ILS  plans,  policy  and  standards, 
component  standardization  guidelines  and  so  on. 

•  HFE  proposal  evaluation  criteria  and  data  are  not  available  and 
source  selections  are  therefore  frequently  made  on  a  subjective  basis. 

Recommendations:  the  following  recommendations  are  made  with  respect  to  the  identi¬ 
fied  technology  gaps  and  shortfalls: 

•  Experimentation  and  literature  review  in  order  to  support  or  provide 
identification  of  needed  change  in  Human  Engineering  standards  and 
specifications 

•  Provisions  for  muiation  and  experimentation  be  established  where 
standard  and  specification  implementation  is  untenable 

•  Establishment  of  formal  functional  allocation  guidelines  in  terms  of 
relative  man-machine  cost  and  reliability  for  operability  and  main¬ 
tainability  design  efforts. 

•  Determination  of  the  feasibility  of  establishing  an  HFE  data  base 
encompassing  all  related  military  standards  and  specifications,  HE 
design  criteria,  NATO  agreements,  etc.,  either  in  printed  and  bound 
or  computer-based  formats. 

•  Determine  the  feasibility  of  deveioping  HFE  proposal  evaluation 
methods  and  criteria. 

4.4  Complex  Workspace  Design 

Technology  advances  and  increased  sophistication  of  major  weapon  systems  in¬ 
creases  the  complexity  of  system  operation  and  workspace  design.  Controls  and  displays, 
such  as  Head  Up  Displays  (HUD),  Voice  Interactive  Systems  (VIS),  potential  for  color 
CRTs,  LEDs,  LCDs,  plasma  displays,  integrated  controls,  and  computer  generated 
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displays,  have  complicated  the  task  of  workspace  design.  For  a  design  to  develop 
workspace  concepts  minimizing  operator  cognitive,  motor,  auditory  and  visual  workload, 
and  maximizing  human  performance  through  workspace  layout,  a  iarge  effort  is  required. 
Constraints  and  controlling  factors  such  as  varying  task  sequences,  environmental 
conditions,  threats,  requirements  for  emergency  egress,  etc.,  further  complicate  the 
effort.  Specific  HFE  technology  shortfalls  relevant  to  the  above  are  such  that: 

•  There  are  no  standardized  methods  which  serve  to  define  the  role  of 
man  in  automated  systems. 

•  Current  computerized  workspace  design  toois  do  not  accommodate 
the  changing  status  of  developing  controls  and  displays,  e.g.,  com¬ 
puter  generated  displays,  multipurpose  displays,  multipurpose  con¬ 
trols. 

•  Computerized  workspace  design  toois  (such  as  'VOLAP  and  CRAFT) 
generally  minimize  visual  and  motor  transition  times  as  a  function  of 
task  sequences  and  control  and  display  criticality.  Other  factors  that 
impact  workspace  layout,  such  as  anthropometry,  control  and  display 
type  and  complexity,  etc,  are  typically  not  addressed. 

A  group  of  computerized  techniques  to  assist  in  generating  crewstation  concepts, 
the  Interactive  Design  Support  Models  (IDS.M),  is  being  developed  at  the  Naval  Air 
Development  Center.  The  interactive  programs  will  address: 

•  Panel  space  allocation  (CUBITS) 

•  Control  and  display  labelling  and  abbreviation  (ABBREV) 

•  Crewstation  assessment  of  reach  (CAR) 

•  Operational  sequences 

•  Functional  grouping  of  controls  and  displays  (GROUP) 

In  addition  to  the  above,  Lewis*  a*  the  Naval  Oceans  Systems  Center  is  doing  extensive 
work  on  the  automation  of  OSDs  using  a  computer  and  graphics  terminal.  A  tool  such  as 
this  could  be  highly  useful  and  beneficial  to  human  engineers,  particularly  as  applied  to 
complex  operations. 

Recommendations:  based  on  the  HFE  shortfalls  identified  and  the  ongoing  effort  at 
NADC,  the  following  recommendations  are  made: 

•  Identify  the  feasibility  of  incorporating  additional  workstation  layout 
factors  in  advanced  or  developing  computerized  toois. 

•  Integrate  computerized  workstation  design  tools  with  computerized 
design  evolution  and  evaluation  tools  (such  as  CAFES,  Siegel- Wolf 
models,  HC5,  etc.) 


*  Personal  communication  with  Mr.  Warren  Lewis. 
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4.5  Maintainability  Design  and  Evaluation 


With  the  increasing  complexity  of  naval  combat  systems  additional  demands  are 
placed  on  system  maintainers,  particularly  In  the  areas  of  diagnostics,  component  access, 
calibration  and  electronics  repair.  Traditionally,  the  role  of  the  human  engineer  has  been 
more  or  less  to  develop  training  systems  given  a  system  configuration  and  to  subject  a 
system  to  an  HFE  maintainability  design  evaluation.  With  respect  to  the  above,  the 
following  HFE  shortfalls  are  identified: 

•  Methods  and  techniques  for  incorporating  HFE  as  part  of  main¬ 
tainability  design  (particularly  in  the  accessibility /arrangements 
area) 

•  Methods  whereby  mockups  are  used  as  part  of  evolutionary  design 
early  in  the  system  configuration 

•  Sufficient  maintainability  design  principles  and  standards  in  terms  of 
generating  maintenance  concepts 

•  HFE  principles  and  standards  related  to  test  bench  design 

Recommendations:  the  basic  recommendation  made  is  to  examine  the  HFE  aspect  of 
maintainability  as  part  of  system  design  and  determine  in  detail  requirements  for  HFE 
methodologies  which  aid  in  generating  maintainability  designs.  Specific  recommendations 
are  as  follows: 

•  Determine  the  feasibility  of  developing  computer-aided  maintaina¬ 
bility  access  and  arrangement  design  tools  similar  to  WOLAP,  etc. 

•  Development  of  maintainability  design  evaluation  tools,  e.g.,  diagnos¬ 
tic  tools  that  radiate  areas  of  required  redesign. 

•  Development  of  maintainability  design  handbook  or  similar  to  address 
issues  such  as: 

-  access  design 

-  test  bench  design 

-  tools  design 

•  Identify  and  develop  principles  and  standards  relevant  to  generating 
maintenance  design  concepts. 

4.6  Test  and  Evaluation 

HFE  T3cE  typically  entails  (1)  assessments  of  the  degree  to  which  a  system's  design 
complies  with  design  criteria  along  various  dimensions  such  as  display  illuminations, 
control  activation  forces,  etc.,  and  (2)  assessments  of  operator/maintainer  performance 
(essentially  relative  to  human  reliability)  of  tasks  and  task  sequencies  given  a  system 
configuration.  With  regard  to  compliance  with  design  criteria,  physical  measurements  are 
usually  taken  (torque  measurements,  etc.)  and  checked  for  compliance  with  standards  such 
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as  MIL-S7D-1472.  The  ultimate  goal  would  be  to  help  optimize  human  performance  via 
standards  compliance.  However,  as  many  systems  cannot  comply  with  standards  (due  to 
constraints  such  as  space,  volume,  weight,  cost),  determinations  ox  totcil  system  reliability 
and  availability  (viewing  human  reliability  as  an  integral  part)  must  be  made  with 
assessments  of  human  performance  in  that  system.  In  many  systems  these  assessments 
are  made  via  direct  observations  of  task  performance  and  operator /maintainer  reporting 
techniques.  Again,  in  some  systems  (notably,  aircraft)  direct  observations  are  not  possible 
(outside  of  mockups  and  simulators  which  cannot  afford  perfect  environment  fidelity)  and, 
hense,  subjective  operator  opinions  are  required.  The  major  HFE  technology  shortfalls 
with  regard  to  the  above  are: 

•  Where  direct  observation  of  task  performance  cannot  be  made,  no 
adequate  techniques  or  methods  were  identified  for  evaluating  these 
systems  . 

•  Insufficient  data  base  for  estimating  operator/maintainer  perfor¬ 
mance  in  lieu  of  standards  compliance. 

The  Mission  Operability  Assessment  Technique  (MOAT)  may  aieviate  the  first  technology 
gap.  MOAT  is  being  developed  at  Pacific  Missile  Test  Center  and  the  Naval  Air 
Development  Center,  and  is  based  in  part  on  the  Functional  Description  Inventory  (FDI  is 
in  fact  a  part  of  MOAT  development). 

Recommendation:  it  is  recommended  that  development  of  MOAT  continue  and  be 
expanded  for  generalized  use  for  systems  other  than  a  single  seat  jet  aircraft. 

4.7  Manning  and  Training 

The  all-volunteer  armed  forces  concept  has  changed  drastically  the  characteristics 
and  availability  of  soldiers,  sailors  and  airmen.  As  a  result,  manning  and  training  have 
become  far  larger  issues  within  the  services  than  previously.  The  following  technology 
shortfalls  are  identified  relevant  to  manning  and  training: 

•  Fewer  techniques  exist  for  deriving  early  estimations  of  system 
specific  manpower  requirements. 

•  There  are  no  standardized  techniques  for  skill  referenced  job  design. 

•  There  are  no  techniques  which  integrate  HFE  design  and  training. 

Relevant  to  the  first  shortfall,  two  methods  which  may  soon  be  available  are  the 
Logistics  Composite  Models  (LCOM)  and  the  SHIPS  II  program.  The  LCOM  model  being 
developed  by  the  Air  Force  Human  Resources  Laboratory  will  be  useful  in  predicting 
maintenance  manpower  requirements  for  aircraft  systems  during  development  (Tetmeyer, 
et  al,  1976).  The  Ship  II  model  being  developed  at  the  Naval  Personnel  Research  and 
Development  Center  is  directed  towards  predicting  total  ship  manning  levels. 
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Recommendations: 

•  It  is  recommended  that  techniques  towards  integrating  HFE  and 
training  system  design  be  developed. 

•  It  is  recommended  that  standard  methods  for  developing  job  designs 
based  on  skill  requirements  be  developed. 


LIST  OF  ACRONYMS 


AIR 

American  Institute  for  Research 

A-OSD 

Automated  Operational  Sequence  Diagram 

APS 

Analytic  Profile  System 

Applied  Psychological  Services 

BAT 

Behavioral  Analysis  of  Tasks 

CAFES 

Computer  Aided  Functional  Allocation  Evaluation  System 

CAFES  CAD 

CAFES  Computer  Aided  Design 

CAFES  CGE 

CAFES  Crewstation  Geometry  Evaluation 

CAFES  DMS 

CAFES  Data  Management  System 

CAFES  FAM 

CAFES  Functional  Allocation  Model 

CAFES  SWAM 

CAFES  Statistical  Workload  Analysis  Model 

CAFES  WAM 

CAFES  Workload  Analysis  Model 

CAR 

Crewstation  Assessment  of  Reach 

CIC 

Combat  Information  Center 

CM 

Corrective  Maintenance 

COMBIMAN 

Computerized  Biomechanical  Man  Model 

CRAFT 

Computerized  Relative  Allocation  of  Facilities  Technique 

CRT 

Cathod  Ray  Tube 

DCP 

Decision  Coordinating  Paper 

DEI 

Display  Evaluation  Index 

DH 

Design  Handbook 

DIDs 

Data  Item  Descriptions 

DoD 

Department  of  Defense 

DP 

Developmental  Proposal 

DNSARC 

Department  of  the  Navy  System  Acquisition  Review  Council 

DSARC 

Defense  System  Acquisition  Review  Council 

DTicE 

Development  Test  3c  Evaluation 

ERU 

Elementary  Reliability  Unit 

ERUPT 

Elementary  Reliability  Unit  Predictive  Technique 

FBD 

Functional  Block  Diagram 

FC 

Flow  Chart 

FDI 

Functional  Description  Inventory 

FFD 

Functional  Flow  Diagram 
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FP3PA 

HE 

HECAD 

HEDGE 

HF 

HFE 

HFTEMAN 

HOS 

HUD 

IDSM 

ILS 

ILSMP 

IPISD 

ISD 

3PA 

3PA  A  ATT  A 

LCD 

LED 

LOR 

MAP 

MENS 

MIL  HDBK 

MIL  STD 

MOAT 

MT3F 

MTTR 

NATO 

03C5 

03T 

OPREDS 

OR 

ORACLE 

OSD 

OTdcE 


Fully  Proceduralized  3ob  Performance  Aids 
Human  Engineering 

Human  Engineering  Computer  Aided  Design 

Human  Engineering  Data  Guide  for  Evaluation 

Human  Factors 

Human  Factors  Engineering 

Human  Factors  Test  dc  Evaluation  Manual 

Human  Operator  Simulator 

Heads  Up  Display 

Interactive  Design  Support  Models 

Integrated  Logistics  Support 

Integrated  Logistics  Support  Master  Plan 

Interservice  Procedures  ISD 

Instructional  Systems  Development 

3ob  Performance  Aid 

3ob  Performance  Aid  Augmented  Action  Tree  Troubleshooting  Aid 
Liquid  Crystal  Display 
Light  Emitting  Diode 
Level  of  Repair 

Means  to  Achieve  Performance 
Mission  Element  Need  Statement 
Military  Handbook 
Military  Standard 

Mission  Operability  Assessment  Technique 
Mean  Time  Between  Failure 
Mean  Time  to  Repair 
North  Atlantic  Treaty  Organization 
Office  of  the  3oint  Chiefs  of  Staff 
On-the-3ob  Training 

Operational  Performance  Recording  and  Evaluation  Data  System 
Operational  Requirement 

Operations  Research  and  Critical  Link  Evaluation 
Office  of  the  Secretary  of  Defense 
Operational  Sequence  Diagram 
Operational  Test  <5c  Evaluation 
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PETER 

PM 

PMP 

PPI 

R&D 

RDT&E 

RFP 

RNO 

SAIM 

SAINT 

SECDEF 

S-OSD 

STO 

SW 

TA 

TA/OSD 

TART 

TEA 

TECEP 

TEPPS 

TEMP 

T&E 

THERP 

TIM 

TLA- 1 

TM 

VIS 

WOLAP 


Performance  Evaluation  Testing  for  Environmental  Research 

Program  tManager 

Program  Master  Plan 

Personnel  Performance  Indices 

Research  6c  Development 

Research,  Development,  Test  <5c  Evaluation 

Requests  for  Proposals 

Remaining  Number  of  Opportunities 

Systems  Analysis  Integration  Model 

Systems  Analysis  of  Integrated  Networks  of  Tasks 

Secretary  of  Defense 

Spacial  OSD 

Science  <5c  Technology  Objectives 

Siegel- Wolf 

Task  Analysis 

Task  Analysis/OSD 

Task  Analysis  Reduction  Technique 

Task  Equipment  Analysis 

Training  Effectiveness,  Cost  Effectiveness,  Prediction  Technique 
Technique  for  Estimating  Personnel  Performance  Standards 
Test  in  Evaluation  Master  Plan 
Test  &  Evaluation 

Technique  for  Human  Error  Rate  Prediction 

Task  Identification  Matrix 

Timeline  Analysis  Program  -  Model  1 

Technical  Manual 

Voice  Interactive  System 

Workspace  Optimization  Layout  and  Planning 
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EXECUTIVE  OFFICE  Cr  THE  PRESIDENT 
CFrTCS  OF  MANAGEMENT  ANO  3UDGET 

WASHINGTON.  O.C.  20903 


April  5,  1976  CIRCULAR  NO.  A-109 

TO  THE  HEADS  OF  EXECUTr/E  DEPARTMENTS  AMD  ESTABLISHMENTS 
SUBJECT:  Major  Sysram  Acquisitions 


1.  Purpose.  This  Circular  establishes  policies,  to  be 
followed  5y  executive  branch  agencies  in  the  acquisition  of 
major  systems. 

2.  Background.  The  acquisition  of  major  systems  by  the 
Federal  Government  constitutes  one  of  the  most  crucial  and 
expensive  activities  performed  to  meet  national  needs.  Its 
impact  is  critical  on  technology ,  or.  the  Nation '  3  economic 
and  fiscal  policies,  and  on  the  accomplishment^of  Government 
agency  missions  in  such  fields  as  defense,  space,  energy  and 
transportation.  For  a  number  of  years,  there  has  been  deep 
concern  over  the  effectiveness  of  the  management  of  major 
system  acquisitions.  The  report  of  the  Commission  on 
Government  Procurement  recommended  basic  changes  to  improve 
the  process  of  acquiring  major  systems.  This  Circular  is 
based  on  executive  branch  consideration  of  the  Commission's 
reccmmendaticns . 

3.  Responsibility.  Each  agency  head  has  the  responsibility 
to  ensure  than  the  provisions  of  this  Circular  are  followed. 
Thi 3  Circular  provides  administrative  direction  to  heads  of 
agencies  and  does  not  establish  and  shall  not  be  construed 
to  create  any  substantive  or  procedural  basis  for  any  person 
to  challenge  any  agency  action  or  inaction  on  the  basis  that 
such  action  was  not  in  accordance  with  this  Circular. 

4.  Coverage .  This  Circular  covers  and  applies  to: 

a.  Management  of  the  acquisition  of  major  systems, 
including:  *  Analysis  of  agency  missions  9  Determination  of 
mission  needs  9  Setting  of  program  objectives  9 
Determination  of  system  requirements  *  System  program 
planning  *  Budgeting  •  Funding  9  Research  9  Engineering  9 
Development  9  Testing  and  evaluation  9  Contracting  9 
Production  9  Program  and  management  control  9  Introduction 
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of  the  system  into  use  or  otherwise  successful  achievement 
of  program  objectives. 

b.  All  programs  for  the  acquisition  of  major  systems 
even  though : 

(1)  The  system  is  one-of-a-kind. 

(2)  The  agency's  involvement  in  the  system  is 
limited  to  the  development  of  demonstration  hardware  for 
optional  use  by  the  private  sector  rather  than  for  the 
agency's  own  use. 

5.  Definitions .  As  used  in  this  Circular: 

a.  Executive  agency  (hereinafter  referred  to  as  agency) 
means  an  executive  ““department,  and  an  independent 
establishment  within  the  meaning  of  sections  101  and  104(1), 
respectively,  of  Title  5,  United  States  Code. 

b.  Acencv  component  means  a  major  organizational 
subdivision  or  an  agency.  For  example:  The  Army,  Navy,  Air 
Force,  and  Defense  Supply  Agency  are  agency  components  of 
the  Department  of  Defense.  The  Federal  Aviation 
Administration,  Urban  Mass  Transportation  Administration, 
and  the  Federal  Highway  Administration  are  agency  components 
of  the  Department  of  Transportation. 

c.  Agency  missions  means  those  responsibilities  for 
meeting  national  needs  assigned  to  a  specific  agency. 

d.  Mission  need  means  a  required  capability  within  an 
agency's  overall  purpose,  including  cost  and  schedule 
considerations . 

e.  Program  objectives  means  me  capability,  cost  and 
schedule  goals  being  sought  by  the  system  acquisition 
program  in  response  to  a  mission  need. 

f.  Program  means  an  organized  set  of  activities 
directed  toward  a  common  purpose,  objective,  or  goal 
undertaken  or  proposed  by  an  agency  in  order  to  carry  out 
responsibilities  assigned  to  it. 

g.  System  design  concept  means  an  idea  expressed  in 
terms  or  general  performance,  capabilities,  and 
characteristics  of  hardware  and  software  oriented  either  to 
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operate  cr  to  be  operated  as  an  integrated  whole  in  meeting 
a  mission  need. 

h.  Major  system  means  that  combination  of  elements  that 
will  function  together  to  produce  the  capabilities  required 
to  fulfill  a  mission  need.  The  elements  may  include,  for 
example,  hardware,  equipment,  software,  construction,  or 
other  improvements  or  real  property.  Major  system 
acquisition  programs  are  those  programs  that  (1)  are 
directed  at  and  critical  to  fulfilling  an  agency  mission, 

(2 )  entail  the  allocation  of  relatively  large  resources,  and 

(3)  warrant  special  management  attention.  Additional 
criteria  and  relative  dollar  threshold-  for  the 
determination  of  agency  programs  to  be  cons  *red  major 
systems  under  the  purview  of  this  Cireu  . ,  may  be 
established  at  the  discretion  of  the  agency  hea 

i.  System  acquisition  process  means  the  .ence  of 
acquisition  activities  starting  from  t- .  agency’s 
reconciliation  of  it3  mission  needs,  with  its  capabilities, 
priorities  and  resources,  and  extending  through  the 
introduction  of  a  system  into  operational  use  or  the 
otherwise  successful  achievement  of  program  objectives. 

j.  Life  cycle  coat  means  the  sum  total  of  the  direct, 
indirect,  recurring,  nonrecurring ,  and  other  related  costs 
incurred,  or  estimated  to  be  incurred,  in  the  design, 
development,  production,  operation,  maintenance  and  support 
of  a  major  system  over  its  anticipated  useful  life  span. 

6.  General  policy .  The  policies  of  this  Circular  are 
designed'  to"  assure  the  effectiveness  and  efficiency  of  the 
process  of  acquiring  major  systems.  They  are  based  on  the 
general  policy  that  Federal  agencies,  when  acquiring  major 
systems,  will: 

a.  Express  need3  and  program  objectives  in  mission 
terms  and  not  equipment  terms  to  encourage  innovation  and 
competition  in  creating,  exploring,  and  developing 
alternative  system  design  concepts. 

b.  Place  emphasis  on  the  initial  activities  of  the 
system  acquisition  process  to  allow  competitive  exploration 
of  alternative  system  design  concepts  in  response  to  mission 
needs. 


(No.  A-109) 


4 


c.  Communicate  with  Congress  early  in  the  system 
acquisition  process  by  relating  major  system  acquisition 
programs  to  agency  mission  needs.  This  communication  should 
follow  the  requirements  of  Office  of  Management  and  Budget 
(OMB)  Circular  No.  A-10  concerning  information  related  to 
budget  estimates  and  related  materials. 

d.  Establish  clear  lines  of  authority,  responsibility, 
and  accountability  for  management  of  major  system 
acquisition  programs,  utilize  appropriate  managerial  levels 
in  decisionmaking,  and  obtain  agency  head  approval  at  key 
decision  points  in  the  evolution  of  each  acquisition 
program. 


e.  Designate  a  focal  point  responsible  for  integrating 
and  unifying  the  system  acquisition  management  process  and 
monitoring  policy  implementation. 

f.  Rely  on  private  industry  in  accordance  with  the 
policy  established  by  OMB  Circular  No.  A-76. 

7.  Major  system  acquisition  management  objectives.  Each 
agency  acquiring  major  systems  should : 

a.  Ensure  that  each  major  system:  Fulfills  a  mission 
need.  Operates  effectively  in  its  intended  environment. 
Demonstrates  a  level  of  performance  and  reliability  that 
justifies  the  allocation  of  the  Nation's  limited  resources 
for  its  acquisition  and  ownership. 

b.  Depend  on,  whenever  economically  beneficial, 
competition  between  similar  or  differing  system  design 
concepts  throughout  the  entire  acquisition  process. 

c.  Ensure  appropriate  trade-off  among  investment  costs, 
ownership  costs,  schedules,  and  performance  characteristics. 

d.  Provide  strong  checks  and  balances  by  ensuring 
adequate  system  test  and  evaluation.  Conduct  3uch  tests  and 
evaluation  independent,  where  practicable,  of  developer  and 
user. 

e.  Accomplish  system  acquisition  planning,  built  on 
analysis  of  agency  missions,  which  implies  appropriate 
resource  allocation  resulting  from  clear  articulation  of 
agency  mission  needs. 
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f.  Tailor  an  acquisition  strategy  for  each  program,  as 
soon  as  the  agency  decides  to  solicit  alternative  system 
design  concepts,  that  could  lead  to  the  acquisition  cf  a  new 
major  system  and  refine  the  strategy  as  the  program  proceeds 
through  the  acquisition  process.  Encompass  test  and 
evaluation  criteria  and  business  management  considerations 
in  the  strategy.  The  strategy  could  typically  include:  * 
Use  cf  the  contracting  process  as  an  important  tool  in  t.e 
acquisition  program  9  Scheduling  of  essential  elements  of 
the  acquisition  process  0  demonstration,  test,  and 
evaluation  criteria  9  Content  of  solicitations  for  proposals 
9  Decisions  on  whom  to  solicit  9  Methods  for  obtaining  and 
sustaining  competition  9  Guidelines  for  the  evaluation  and 
acceptance  or  rejection  of  proposals  9  Gcals  for  design-no- 
cost  9  Methods  for  projecting  life  cycle  costs  9  Use  of  data 
rights  9  Use  of  warranties  9  Methods  for  analyzing  and 
evaluating  contractor  and  Government  risks  9  Need  for 
developing  contractor  incentives  9  Selection  of  the  type  of 
contract  best  suited  for  each  stage  in  the  acquisition 
process  9  Administration  of  contracts. 

c.  Maintain  a  capability  to:  9  Predict,  review,  assess, 
negotiate  and  monitor  costs  for  system  development, 
engineering,  design,  demonstration,  tasz,  production, 
operation  and  support  (i.e.,  life  cycle  costs)  9  Assess 
acquisition  cost,  schedule  and  performance  experience 
against  predictions,  and  provide  such  assessments  for 
consideration  by  the  agency  head  at  key  decision  points  9 
Make  new  assessments  where  significant  costs,  schedule  cr 
performance  variances  occur  9  Estimate  life  cycle  costs 
during  system  design  concept  evaluation  and  selection,  full- 
scale  development,  facility  conversion,  and  procucticn,  to 
ensure  appropriate  trade-offs  among  investment  costs, 
ownership  costs,  schedules,  and  perfcrmar.ce  9  Use 
independent  cost  estimates,  where  feasible,  for  comparison 
purposes . 

3.  Management  structure. 


a.  The  head  of  each  agency  that  acquires  major  systems 
will  designate  an  acquisition  executive  to  integrate  and 
unify  the  management  process  for  the  agency's  major  system 
acquisitions  and  to  monitor  implementation  of  the  policies 
and  practices  set  forth  in  this  Circular. 

b.  Each  agency  that  acquires — or  is  responsible  for 
activities  leading  to  the  acquisition  cf — major  systems  will 
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establish  clear  lines  of  authority,  responsibility,  and 
accountability  for  management  of  its  major  system 
acquisition  programs. 

c.  Each  agency  should  preclude  management  layering  and 
placing  reporting  procedures  and  paperwork  requirements  on 
program  managers  and  contractors. 

d.  A  program  manager  will  be  designated  for  each  of  the 
agency's  major  system  acquisition  programs.  This 
designation  should  be  made  when  a  decision  is  made  to 
fulfill  a  mission  need  by  pursuing  alternative  system  design 
concepts.  It  is  essential  that  the  program  manager  have  an 
understanding  of  user  needs  and  constraints,  familiarity 
with  development  principles,  and  requisite  management  skills 
and  experience.  Ideally,  management  skills  and  experience 
would  include:  *  Research  and  development  8  Operations  0 
Engineering  8  Construction  °  Testing  0  Contracting  8 
Prototyping  and  fabrication  of  complex  systems  0  Production 
8  Business  8  Budgeting  8  Finance.  with  satisfactory 
performance,  the  tenure  of  the  program  manager  should  be 
long  enough  to  provide  continuity  and  personal 
accountability . 

e.  Upon  designation,  the  program  manager  should  be 
given  budget  guidance  and  a  written  charter  of  his 
authority,  responsibility,  and  accountability  for 
accomplishing  approved  program  objectives. 

f.  Agency  technical  management  and  Government 
laboratories  should  be  considered  for  participation  in 
agency  mission  analysis,  evaluation  of  alternative  system 
design  concepts,  and  support  of  all  development,  test,  and 
evaluation  efforts. 

g.  Agencies  are  encouraged  to  work  with  each  other  to 
foster  technology  transfer,  prevent  unwarranted  duplication 
of  technological  efforts,  reduce  system  costs,  promote 
standardization,  and  help  create  and  maintain  a  competitive 
environment  for  an  acquisition. 

9.  Key  decisions .  Technical  and  program  decisions  normally 
will  be  5ia3e  at  the  level  of  the  agency  component  or 
operating  activity.  However,  the  following  four  key 
decision  points  should  be  retained  and  made  by  the  agency 
head: 
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a.  Identification  and  definition  of  a  specific  mi3sion 
need  to  be  fulfilled,  the  relative  priority  assigned  within 
the  agency,  and  the  general  magnitude  of  resources  that  may 
be  invested. 

b.  Selection  of  competitive  system  design  concepts  to 
be  advanced  to  a  test/demonstration  phase  or  authorization 
to  proceed  with  the  development  of  a  noncompetitive  (single 
concept)  system. 

c.  Commitment  cf  a  system  to  full-scale  development  and 
limited  production. 

d.  Commitment  of  a  system  to  full  production. 

10.  Determination  of  mission  needs . 

a.  Determination  of  mission  need  should  be  based  on  an 
analysis  of  an  agency's  mission  reconciled  with  overall 
capabilities,  priorities  and  resources.  When  analysis  of  an 
agency's  mission  shows  that  a  need  for  a  new  major  system 
exists,  such  a  need  should  not  be  defined  in  equipment 
terms,  but  should  be  defined  in  terms  of  the  mission, 
purpose,  capability,  agency  components  involved,  schedule 
and  cost  objectives,  and  operating  constraints.  A  mission 
need  may  result  from  a  deficiency  in  existing  agency 
capabilities  or  the  decision  to  establish  new  capabilities 
in  response  to  a  technologically  feasible  opportunity. 
Mission  needs  are  independent  of  any  particular  system  or 
technological  solution. 

b .  Where  an  agency  has  more  than  one  component 
involved,  the  agency  will  assign  the  roles  and 
responsibilities  of  each  component  at  the  time  of  the  first 
key  decision.  The  agency  may  permit  two  or  more  agency 
components  to  sponsor  competitive  system  design  concepts  in 
order  to  foster  innovation  and  competition. 

c.  Agencies  should,  as  required  to  satisfy  mission 
responsibilities,  contribute  to  the  technology  base, 
effectively  utilizing  both  the  private  sector  and  Government 
laboratories  and  in-house  technical  centers,  by  conducting, 
supporting,  or  sponsoring:  9  Research  •  System  design 
concept  studies  9  Proof  of  concept  work  9  Exploratory 
subsystem  development  9  Tests  and  evaluations.  Applied 
technology  efforts  oriented  to  system  developments  should  be 
performed  in  response  to  approved  mission  needs. 
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II.  Alternative  systems . 

a.  Alternative  system  design  concepts  will  be  explored 
within  the  context  of  the  agency's  mission  need  and  program 
objectives — with  emphasis  cn  generating  innovation  and 
conceptual  competition  from  industry.  Benefits  to  be 
derived  should  be  optimized  by  competitive  exploration  of 
alternative  system  design  concepts,  and  trade-offs  of 
capability,  schedule,  and  cost.  Care  should  be  exercised 
during  the  initial  steps  of  the  acquisition  process  not  to 
conform  mission  needs  or  program  objectives  to  any  known 
systems  or  products  that  might  foreclose  consideration  of 
alternatives . 

b.  Alternative  system  derigr.  concepts  will  be  solicited 
from  a  broad  base  of  qualified  firms.  In  order  to  achieve 
the  most  preferred  system  solution,  emphases  will  be  placed 
cn  innovation  and  competition.  To  this  and-  participation 
of  smaller  and  newer  businesses  should  be  encouraged. 
Concepts  will  be  primarily  solicited  from  private  industry; 
and  when  beneficial  to  the  Government,  foreign  technology, 
and  equipment  may  be  considered. 

c.  Federal  laboratories,  federally  funded  research  and 
development  centers,  educational  institutions,  and  other 
not-for-profit  organisations  may  also  be  considered  as 
sources  fer  competitive  system,  design  concepts.  Ideas, 
concepts,  or  technology,  developed  by  Government 
laboratories  or  at  Government  smpa r.,e,  may  be  made  available 
tc  private  industry  through  the  procurement  process  or 
through  other  established  procedures.  Industry  proposals 
may  be  made  or.  the  oasis  of  tna.se  ideas,  concepts,  and 
technology  or  or.  the  basis  of  feasible  alternatives  which 
the  propcoer  considers  superior. 

d.  Research  a  r.A  fare  i  content  erf  ruts  should  emphasize 
u.'.rly  competitive  exploration  or  a  1  terra  rives ,  as  relatively 
inexpensive  insurance  ageins:.  prara~v.r=  t“  preordained 
choice  of  a  system  that  may  prove  cn  be  eit.-.c*  mere  costly 
or  less  effective. 

e.  Requests  for  alt-  stive  system  i-cign  concept 
proposals  will  explain  ne  ■■us5:.:r  need,  schedule,  cost, 
capability  objectives,  operating  constraints.  Each 
offeror  will  oe  free  tc  .c nose  ris  own  technical  approach, 

:  air.  design  features,  subsystems,  ana  alternatives  to 
-■cheduie,  cost,  and  ca~ab:-. tty  .-cals  Ir.  the  conceptual  and 
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less  than  full- soala  d3-»"2_cp5ua 
not  be  restricted  by  detailed 
standards . 


i  stages .  contractors  should 
Government  specifications  anc 


-  -  Selections  from  ccmtating  system  design  concept 
proposals  will  be  based  on  a  review  by  a  team  of  experts, 
preferably  from  inside  and  outside  the  responsible  component 
development  organisation,  ;h..zh  a  review  will  consider:  !i) 
Proposed  systam  functional  and  performance  capabilities  to 
meet  mission  needs  and  program  objectives,  including 
resources  required  and  benefits  so  be  derived  by  trade-off 3, 
where  feasible,  among  technical  performance,  acquisition 
costs,  ownership  certs ,  time  to  develop  and  orocura;  and  (2) 
The  relevant  accomplishment  record  of  conpetitors. 

g.  During  she  uncertain  period  of  identifying  and 
exploring  alternative  system,  design  concepts,  contracts 
covering  relatively  short  tine  periods  at  planned  dollar 
levels  will  be  used.  Timely  technical  reviews  of 
alternative  system  casicn  concepts  will  be  made  to  affect 
the  orderly  elimination  of  these  least  attractive. 


h. .  Contractors  should  be  provided  with  operational  test 
conditions,  mission  performance  crizaria,  and  life  cycle 
cost  factors  that  will  be  used  by  the  agency  in  the 
evaluation  and  selection  of  the  system (s)  for  full-scale 
development  and  production  - 


program. 

other 

concept 


The  participating  contractors  should  be  provided 
levar.z  cparatior.ai  and  support  experience  through  the 
manager,  as  necessary,  in  developing  performance  and 
requirements  for  each  alternative  system  design 
as  test3  and  trade-offs  are  made. 


^2-  ^Development  of  subsystems  that  are  intended  to  be 
included  in  a  major  system  acquisition  program  will  be 
restricted  to  less  than  fully  designed  hardware  (full-scale 
development)  until  the  subsystem  is  identified  as  a  part  of 
*  system  candidate  for  fuil-".caie  development.  Exceptions 
fliay  oe  authorized  by  the  agency  head  if  the  subsystems  are 
lei:-:  lead  time  items  that  fulfill  a  recognized  generic  need 
or  --  they  have  a  high  potential  for  common  use  among 
several  existing  or  future  svstems. 
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12 .  Demonstrations . 


a.  Advancement  to  a  competitive  tast/demonstratior. 
phase  may  be  approved  when  the  agency's  mission  need  and 
program  objectives  are  reaffirmed  and  when  alternative 
system  design  concepts  are  selected. 

b.  Major  system  acquisition  programs  will  be  structured 
and  resources  planned  to  demonstrate  and  evaluate  competing 
alternative  system  design  concepts  that  have  been  selected. 
Exceptions  may  be  authorized  by  the  agency  head  if 
demonstration  is  not  feasible. 

c.  Development  of  a  single  system  design  concept  that 
has  not  been  competitively  selected  should  be  considered 
only  if  justified  by  factors  such  as  urgency  of  need,  or  by 
the  physical  and  financial  impracticality  of  demonstrating 
alternatives.  Proceeding  with  the  development  of  a 
noncompetitive  (single  concent)  system  may  be  authorised  by 
the  agency  head.  Strong  agency  program  management  and 
technical  direction  should  be  used  for  systems  that  have 
been  neither  competitivelv  selected  nor  demonstrated. 

13 .  Full-scale  development  and  production . 

a.  Ful.l -scale  development,  including  limited 
production,  may  be  approved  when  the  agency's  mission  need 
and  program  objectives  are  reaffirmed  and  competitive 
demonstration  results  verify  that  the  chosen  system  design 
concept (s)  is  sound. 

b.  Full  production  may  be  approved  when  the  agency's 
mission  need  and  program  objectives  are  reaffirmed  and  when 
svsoam  performance  has  been  satisfactorily  tested, 
independent  of  the  agency  development  and  user 
organizations,  and  evaluated  in  an  environment  that  assures 
demonstration  in  expected  operational  conditions. 
Exceptions  to  indeper-'.sn-  tasting  may  be  authorized  by  the 
agency  head  under  suon  circumstances  as  physical  or 
financial  impracticabf lit}  nr  extreme  urgency. 

c.  Selection  of  a  sy;tam(s)  and  contractor (s)  for  full- 
scale  development  and  prc  '.action  is  to  be  made  on  the  basis 
of  (1)  system  performance  measured  against  current  mission 
need  and  program  objectives,  (1)  an  evaluation  of  estimated 
acquisition  and  ovniej.  -hip  costs,  and  (3)  such  factors  as 
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contractor  (s)  demonstrated  management ,  financial,  and 
technical  capabilities  to  meet  precram  objectives. 

d.  The  program  manager  will  monitor  sy3tair.  test3  and 
contractor  progress  in  fulfilling  system  performance,  cost, 
3rd  schedule  commitments.  Significant  actual  cr  forecast 
variances  will  be  brought  to  the  attention  of  the 
appropriate  management  authority  for  corrective  action. 

14.  Budgeting  and  financing .  Beginning  with  FY  1973  all 
agencies  will ,  as  part  cf  the  budget  process,  present 
budgets  in  term s  of  agency  missions  in  consonance  with 
Section  201(1}  cf  the  Budget  ar.d  Accounting  Act,  1921,  as 
added  by  Section  5  Cl  cf  the  Congressional  Budget  Act  of 
1374,  and  in  accordance  with  CMB  Circular  A-ll.  In  so 
doing,  the  agencies  are  desired  to  sepaxately  identify 
research  and  development  funding  for:  (1)  The  general 
technology  base  in  support  of  the  agency's  overall  missions, 
(?)  The  specific  development  efforts  ir.  support  of 
alternative  system  design  concepts  to  accomplish  each 
miss  ten  need,  and  (2;  Full-scale  developments.  Each  agency 
should  ensure  that  research  arc  development  is  net 
undesirably  duplicated  across  its  missions. 

15.  Information  to  Congress. 


a.  Troceduras  for  this  purpose  vrili  be  developed  in 
conjunction  with  the  Office  of  Management  and  Budget  and  the 
various  committees  of  Congress  having  oversight 
responsibility  for  agency  activities.  Beginning  with  FY 
1379  budget  each  agency  will  inform  Congress  in  the  normal 
budget  process  abcut  agency  missions,  capabilities, 
deficiencies,  and  needs  and  objectives 
acquis i tier,  programs,  in  consonance  tith  Sect! 
the  Congressional  Budget  Act  cf  1374. 


b.  Ciscicsure 
proceed  with  a 


:f  the  basis  for  an  agency 
single  system  design 
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a.  Policy  directives,  regulations,  and  guidelines  as 
they  are  issued. 

b.  Within  six  months  after  the  date  of  this  Circular,  a 
time-phased  action  plan  for  meeting  the  requirements  of  this 
Circular. 

c.  Periodically,  the  agency  approved  exceptions 
permitted  under  the  provisions  of  this  Circular. 

This  information  will  be  used  by  the  OMB,  in  identifying 
major  system  acquisition  trends  and  in  monitoring 
implementations  of  this  policy. 

13 .  Inquiries .  All  questions  or  inquiries  should  be 
submitted  to  the  OMB,  Administrator  for  Federal  Procurement 
Policy.  Telephone  number,  area  code,  202-395-4677. 


ADMINISTRATOR  FOR 
FEDERAL  PROCUREMENT  POLICY 
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